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MEMORANDUM  FOR  UNDER  SECRETARY  OF  DEFENSE  FOR  ACQUISITION 
ASSISTANT  SECRETARY  OF  THE  AIR  FORCE 
(FINANCIAL  MANAGEMENT  AND  COMPTROLLER) 


SUBJECT:  Audit  Report  on  Selected  Acquisition  Actions  on  the 
C-17  Aircraft  (Report  No.  91-007  ) 


This  is  our  final  report  on  the  Audit  of  Selected 
Acquisition  Actions  on  the  C-17  Aircraft.  We  made  the  audit  from 
October  24,  1989,  through  February  2,  1990,  in  response  to  a  DoD 
Hotline  allegation  that  the  Air  Force  inappropriately  exercised  a 
contract  option  in  July  1989  to  buy  four  C-17  aircraft  for 
$691  million  and  that  the  status  of  C-17  software  development  was 
worsening. 

The  complainant  alleged  that  not  all  software  was  reviewed 
during  the  mission  computer  software  critical  design  review  (CDR) 
conducted  in  April  1989  and  that  not  all  deficiencies  identified 
during  the  CDR  were  corrected  before  the  contract  option  was 
exercised.  The  complainant  also  alleged  that  changes  made  in 
July  1989  regarding  software  development  responsibilities  of 
two  subcontractors  indicated  that  the  status  of  software 
development  was  worsening  and  that  the  prime  contractor,  Douglas 
Aircraft  Company  (Douglas),  McDonnell  Douglas  Corporation, 
planned  to  replace  a  third  subcontractor  late  in  the  program. 

The  objectives  of  the  audit  were  to  review  the  circumstances 
and  documentation  relating  to  the  C-17  aircraft  that  were 
procured  in  July  1989  and  to  assess  the  status  of  software 
development  for  selected  C-17  subsystems.  We  also  evaluated 
whether  the  program  development  schedule  was  realistic.  In 
addition,  we  evaluated  compliance  with  internal  controls 
applicable  to  the  C-17  program. 

The  C-17  is  being  developed  to  provide  additional  capability 
to  the  Air  Force  to  airlift  all  classes  of  military  cargo  and  to 
operate  into  short,  austere  airfields.  At  the  time  of  our  audit, 
the  Air  Force  planned  to  buy  210  C-17  aircraft  at  a  total  cost  of 
about  $41.8  billion.  In  April  1990,  due  to  reductions  in  the  DoD 
budget  and  the  expectations  of  greater  warning  of  a  possible 
attack,  the  Secretary  of  Defense  planned  to  reduce  the 
procurement  quantity  to  120  aircraft  at  a  total  cost  of  about 
$29.9  billion.  The  Air  Force  expects  the  first  flight  of  the 
C-17  to  occur  in  June  1991. 
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The  audit  showed  that  Douglas  did  not  review  the  detail 
design  of  all  software  for  the  mission  computer  during  the  CDR  in 
April  1989.  The  full-scale  development  (FSD)  contract  made  the 
exercise  of  the  contract  option  for  FY  1989  contingent  on  the 
determination  by  the  procuring  contracting  officer  at  the  C-17 
System  Program  Office  that  the  mission  computer  CDR  was 
completed.  The  FSD  contract  allowed  the  C-17  Program  Director  to 
exercise  the  contract  option  for  FY  1989  at  any  time, 
notwithstanding  CDR  results.  However,  we  believe  that  the 
April  1989  CDR  did  not  provide  a  sufficient  basis  to  determine 
that  the  mission  computer  CDR  was  completed  since  not  all  of  the 
mission  computer  detail  design  for  software  was  reviewed.  We 
determined,  nonetheless,  that  plans  prepared  by  Douglas  were 
adequate  to  correct  the  critical  deficiencies  identified  during 
the  CDR. 

The  audit  showed  that  serious  problems  had  been  experienced 
in  developing  software  for  the  C-17  program,  but  that  Douglas  had 
taken  actions  to  keep  the  software  development  program  from 
worsening.  Douglas  planned  to  obtain  a  new  subcontractor  for  a 
hardware  panel  for  an  avionics  subsystem,  but  we  believe  that  the 
change  will  not  adversely  affect  the  program  development 
schedule.  The  Air  Force's  event-based  contract  award  strategy  is 
an  excellent  method  to  ensure  that  procurement  decisions  are 
justified  and  not  predetermined.  However,  the  Air  Force  needed 
to  clarify  the  provisions  of  the  FY  1990  milestone  event  to 
ensure  that  adequate  development  progress  is  demonstrated  before 
aircraft  are  procured  in  FY  1990. 

The  audit  also  showed  that  delays  in  the  development  and 
integration  of  software  were  major  risks  to  the  program 
development  schedule,  and  further  schedule  delays  could  yet 
occur.  Douglas  has  initiated  actions  to  improve  the  management 
of  the  C-17  program;  however,  because  the  actions  were  taken  late 
in  the  program  and  have  not  yet  proved  to  be  successful,  further 
delays  in  the  program  schedule  may  occur. 

The  results  of  the  audit  are  summarized  in  the  following 
paragraphs,  and  the  details,  audit  recommendations,  and 
management  comments  are  in  Part  II  of  this  report.  The  audit  did 
not  identify  any  material  internal  control  weaknesses  or 
potential  monetary  benefits;  however,  other  benefits  are 
summarized  in  Appendix  E. 

Before  the  mission  computer  software  CDR  was  completed,  as 
required  by  a  November  1988  modification  of  the  FSD  contract  with 
Douglas,  the  C-17  System  Program  Office  (SPO)  exercised  a 
contract  option  to  buy  four  C-17  aircraft.  The  C-17  Program 
Director  considered  the  CDR  to  be  completed  because  the  top-level 
software  design  of  the  mission  computer  was  reviewed  during  the 
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CDR,  and  Douglas  had  taken  management  actions  to  ensure 
completion  of  the  detail  design  of  the  mission  computer  software. 
Military  Standard  1521A,  "Technical  Reviews  and  Audits  for 
Systems,  Equipments,  and  Computer  Software,"  states  that  a 
software  CDR  shall  be  conducted  when  the  detail  design  is 
essentially  complete.  When  the  CDR  was  conducted  in  April  1989, 
detail  design  was  completed  for  only  the  software  needed  for  the 
aircraft's  first  100  hours  of  flight  testing,  or  about  60  percent 
of  all  software  in  the  mission  computer.  In  our  opinion,  the 
April  1989  CDR  only  when  taken  together  with  a  software  design 
review  to  be  done  in  late  1990  constitutes  a  completed  CDR  of  the 
mission  computer  software.  We  believe  that  if  experts  in 
software  developments  had  attended  meetings  of  the  Conventional 
Systems  Committee  (the  Committee)  of  the  Defense  Acquisition 
Board  regarding  the  C-17  aircraft,  the  Committee  would  have 
questioned  more  rigorously  Douglas's  ability  to  complete  the  CDR 
in  April  1989.  The  C-17  program  will  be  delayed  if  the  software 
is  not  fully  developed  when  required  for  operational  testing.  We 
recommended  that  individuals  with  technical  expertise  attend 
Committee  meetings  when  their  oversight  of  software  development 
matters  is  needed.  We  also  recommended  that  the  FSD  contract  be 
amended  to  require  completion  of  the  software  design  review 
before  procuring  four  C-17  aircraft  for  FY  1990  and  to  specify 
the  minimum  assembly  required  for  the  test  aircraft  for  the 
milestone  event  for  the  Lot  III  production  contract  (page  7). 

Douglas  had  not  met  contractual  milestones  for  developing 
the  aircraft  and  had  not  prepared  an  adequate  plan  for  testing 
and  integrating  C-17  avionics  software.  The  Air  Force  had  to 
postpone  major  program  milestones,  and  further  delays  in 
development  may  occur.  We  recommended  that  OSD  establish 
procurement  policy  that  requires  contractors  to  submit  a  systems 
engineering  management  plan  as  part  of  the  FSD  proposal  for 
contracts  that  assign  total  performance  responsibility  to 
contractors.  In  addition,  we  recommended  that  the  Air  Force  set 
a  deadline  for  Douglas  to  prepare  an  avionics  integration  test 
plan  and  require  Douglas  to  prepare  a  revised  schedule  showing 
new  program  milestone  dates.  We  also  recommended  that  the  Air 
Force  obtain  compensation  from  Douglas  for  missed  contractual 
milestones.  We  are  not  claiming  potential  monetary  benefits  for 
any  compensation  received  from  Douglas.  The  intent  of  the 
recommendation  was  to  add  impetus  to  the  Air  Force's  efforts  in 
obtaining  compensation,  which  began  before  the  audit  was 
initiated.  During  the  audit,  we  were  informed  that  Douglas  had 
not  responded  to  a  request  from  the  SPO  dated  August  18,  1989,  to 
provide  compensation  for  missed  contractual  milestones  (page  17). 

A  draft  of  this  report  was  provided  to  the  Under  Secretary 
of  Defense  for  Acquisition  and  the  Assistant  Secretary  of  the  Air 
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Force  (Financial  Management  and  Comptroller)  on  June  14,  1990. 
Comments  were  received  from  the  Office  of  the  Under  Secretary  on 
August  20,  1990,  and  from  the  Air  Force  on  August  17,  1990. 

The  Office  of  the  Under  Secretary  of  Defense  for  Acquisition 
nonconcurred  in  Recommendation  A.I.,  which  would  have  required 
the  Committee  to  have  two  permanent  members  with  expertise  in 
software  matters.  The  management  comments  stated  that  the 
Committee's  policy  has  been  to  allow  attendance  by  supporting 
staff  members  with  relevant  expertise.  We  accepted  the  premise 
that  principals  will  bring  individuals  with  software  expertise  to 
future  Committee  meetings  when  their  expertise  is  required,  and 
we  revised  our  recommendation  in  the  final  report  accordingly. 
We  request  management  comments  on  the  revised  recommendation. 
Management  nonconcurred  in  Recommendation  B.I.,  which  recommended 
that  procurement  policy  be  established  for  contracts  for  major 
acquisition  programs  that  assign  total  performance  responsibility 
to  contractors,  because  broad  guidance  on  systems  engineering 
currently  exists  in  the  draft  DoD  Instruction  5000.2,  and  because 
it  was  not  clear  that  problems  with  developing  and  integrating 
C-17  avionics  systems  would  have  been  prevented  by  a  policy 
requiring  formal  documentation  of  systems  engineering.  We 
believe  the  recommendation  is  still  warranted  for  the  reasons 
discussed  in  Part  II  of  this  report. 

The  Air  Force  nonconcurred  in  all  recommendations  (A. 2. a., 
A. 2 .b. ,  B.2 .a. ,  B.2.b.,  and  B.2.c.)  addressed  to  it.  Although 
the  Air  Force  nonconcurred,  the  actions  taken  in  response  to 
Recommendation  A.2.b.,  which  recommended  a  definition  be  made  of 
the  minimum  assembly  required  to  consider  the  test  aircraft 
assembled,  and  actions  taken  in  response  to 
Recommendation  B.2.b.,  which  recommended  a  revised  program 
schedule,  meet  the  intent  of  the  audit  recommendations. 
Accordingly,  additional  management  comments  on  Recommendations 
A.2.b.  and  B.2.b.  in  the  final  report  are  not  required. 

The  Air  Force  nonconcurred  in  Recommendation  A. 2. a.,  which 
would  have  required  the  mission  computer  software  CDR  to  be 
completed  before  awarding  the  Lot  III  production  contract.  The 
Air  Force  and  Douglas  consider  the  mission  computer  CDR  to  be 
completed,  as  evidenced  by  the  exercise  of  the  FY  1989  contract 
option.  Accordingly,  we  revised  our  recommendation  in  the  final 
report  to  require  that  the  software  design  review  of  the  mission 
computer  software  be  completed  before  awarding  the  FY  1990 
production  contract.  We  request  management  comments  on  the 
revised  recommendation.  The  Air  Force  also  nonconcurred  in 
Recommendation  B.2. a.,  which  recommended  setting  a  completion 
date  for  Douglas  to  finalize  and  issue  an  avionics  integration 
test  plan,  because  avionics  planning  is  an  evolutionary  process, 
and  updated  versions  of  an  Avionics  Integration  Test  Plan  were 
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submitted  by  Douglas  in  April  1990  and  in  July  1990.  In  addition, 
the  Air  Force  nonconcurred  in  Recommendation  B.2.C.,  to  obtain 
compensation  to  the  Government  from  Douglas  for  missed 
contractual  milestones,  because  the  process  to  obtain  the 
compensation  had  begun  before  the  audit  was  initiated.  We 
believe  Recommendations  B.2.a.  and  B.2.c.  are  still  warranted  for 
the  reaspns  discussed  in  Part  II  of  the  report. 

DoD  Directive  7650.3  requires  that  all  audit  recommendations 
be  resolved  within  6  months  of  the  date  of  the  final  report. 
Accordingly,  the  Under  Secretary  of  Defense  for  Acquisition  and 
the  Air  Force  should  provide  final  comments  on  the  unresolved 
issues  in  this  report  within  60  days  of  the  date  of  this 
memorandum. 

The  courtesies  extended  to  the  audit  staff  are  appreciated. 
A  list  of  audit  team  members  is  in  Appendix  G.  Copies  of  this 
report  are  being  provided  to  the  activities  listed  in  Appendix  H. 
If  you  have  any  questions  on  this  audit,  please  contact 
Mr.  Ronald  Porter  on  (703)  693-0163  (AUTOVON  223-0163)  or 
Mr.  John  Mundell  on  (703)  693-0168  (AUTOVON  223-0168). 


Robert  J .  I^reberman 
Assistant  Inspector  General 
for  Auditing 


Enclosure 

cc: 

Secretary  of  the  Air  Force 
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SELECTED  ACQUISITION  ACTIONS  ON  THE  C-17  AIRCRAFT 


PART  I  -  INTRODUCTION 


Background 

The  C-17  aircraft  is  being  developed  to  provide  additional 
capability  to  the  Air  Force  to  airlift  the  full  range  of  DoD 
cargo  and  to  provide  military  capabilities  currently  not 
available  in  any  one  cargo  aircraft.  The  C-17  aircraft  would 
help  to  attain  a  fiscally  constrained  goal  established  in. 1981 
for  an  intercontinental  airlift  of  66  million  ton-milesA'  per 
day.  The  C-17  is  being  designed  to  land  on  austere  airfields  as 
short  as  3,000  feet  to  deliver  cargo  directly  to  user  forces. 

A  DoD  Hotline  complaint  alleged  that  the  Air  Force 
inappropriately  exercised  a  contract  option  in  July  1989  to  buy 
four  C-17  aircraft  for  $691  million  and  that  the  status  of  C-17 
software  development  was  worsening.  The  full-scale  development 
(FSD)  contract  with  Douglas  Aircraft  Company  (Douglas),  McDonnell 
Douglas  Corporation,  required  the  procuring  contracting  officer 
for  the  C-17  System  Program  Office  to  determine  that  the  critical 
design  review  (CDR)  of  the  mission  computer  was  complete  before 
exercising  the  production  option  for  FY  1989.  The  complainant, 
who  was  granted  anonymity,  alleged  that  not  all  of  the  software 
for  the  mission  computer  was  reviewed  during  the  CDR  conducted 
April  10  to  14,  1989.  It  was  further  alleged  that  certain 
deficiencies  identified  during  the  CDR  were  not  corrected  before 
the  FY  1989  procurement  option  was  exercised.  In  addition,  the 
complainant  alleged  that  Douglas's  replacement  of  the 
subcontractor  for  the  electronic  flight  control  system  and 
assumption  of  responsibility  for  the  software  for  the  mission 
computer  from  another  subcontractor  indicated  that  the  status  of 
software  development  was  worsening.  The  complainant  also  alleged 
that  at  a  late  point  in  the  program,  Douglas  planned  to  replace 
another  subcontractor  that  was  developing  an  avionics  subsystem. 
The  complainant's  overriding  concern  was  that  the  management  of 
the  software  development  process  by  the  Air  Force  and  Douglas 
appeared  to  increase  the  risk  that  the  C-17  aircraft  would  not  be 
developed  on  time  and  in  an  economical  manner,  and  that  expensive 
retrofits  of  the  aircraft  might  be  required  after  aircraft 
delivery. 

At  the  time  of  our  audit,  the  Air  Force  planned  to  buy  210  C-17 
aircraft  at  a  total  cost  of  about  $41.8  billion.  In  April  1990, 
due  to  reductions  in  the  DoD  budget  and  expectations  of  greater 
warning  of  a  possible  attack,  the  Secretary  of  Defense  planned  to 
reduce  the  procurement  quantity  to  120  C-17  aircraft  at  a  total 


i/  a  ton-mile  is  one  ton  of  cargo  flown  one  mile. 
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cost  of  $29.9  billion.  The  FSD  contract  with  Douglas  provides 
for  the  design,  development,  and  testing  of  the  aircraft  and  the 
production  of  two  ground  test  aircraft  and  one  flight  test 
aircraft.  The  fixed-price  incentive,  firm-target  contract  has  a 
target  price  of  $4.2  billion.  In  addition,  the  FSD  contract 
includes  two  production  options.  The  first  option  for 
two  aircraft  (Lot  I)  was  exercised  on  January  20,  1988,  for 
$604  million.  The  second  option  for  four  aircraft  (Lot  II)  was 
exercised  on  July  28,  1989,  for  $691  million.  As  of  the  time  our 
audit  concluded,  the  Air  Force  was  preparing  to  negotiate  a 
production  contract  to  acquire  additional  C-17  aircraft.  The 
last  C-17  will  be  procured  in  FY  1997. 

Objectives  and  Scope 

The  objectives  of  our  audit  were  to  review  the  circumstances  and 
documentation  relating  to  the  contract  option  exercised  in 
July  1989  to  buy  four  C-17  aircraft  for  $691  million  from  Douglas 
and  to  assess  the  status  of  software  development  for  selected 
C-17  subsystems.  We  also  evaluated  whether  the  program 
development  schedule  was  realistic.  During  our  audit,  we 
obtained  information  on  the  CDR  done  in  April  1989  for  the  C-17 
mission  computer  software  and  on  the  development  of  software  for 
selected  C-17  subsystems.  We  examined  records  of  C-17  program 
reviews  dated  from  August  1988  through  February  1990  and 
documentation  dated  from  December  1985  through  February  1990, 
which  included  program  plans,  development  schedules,  and  the  C-17 
FSD  contract.  We  interviewed  officials  at  OSD,  the  Air  Force, 
Douglas,  and  Douglas’s  subcontractors  involved  in  the  C-17 
acquisition  program.  Our  technical  assessment  staff  assisted  the 
auditors  in  evaluating  systems  engineering  and  software 
development  efforts  and  results,  C-17  program  contracting 
practices,  and  software  quality  assurance  procedures.  Although 
we  focused  the  scope  of  our  audit  primarily  on  software 
development  and  on  whether  the  mission  computer  software  CDR  was 
completed,  we  noted  some  hardware  development  problems  that  are 
discussed  in  Part  II  of  the  report. 

This  economy  and  efficiency  audit  was  conducted  from  October  24, 
1989,  through  February  2,  1990,  in  accordance  with  auditing 
standards  issued  by  the  Comptroller  General  of  the  United  States, 
as  implemented  by  the  Inspector  General,  DoD,  and  accordingly, 
included  such  tests  of  internal  controls  as  were  considered 
necessary.  The  activities  we  visited  during  the  audit  are  listed 
in  Appendix  F. 

Internal  Controls 


We  reviewed  internal  controls  relating  to  our  audit  objectives. 
In  assessing  internal  controls,  we  evaluated  control  techniques 
such  as  management  plans,  written  policies,  design  reviews. 
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contractual  provisions,  and  independent  software  reviews.  No 
material  internal  control  weaknesses  were  identified  during  the 
audit. 

Prior  Audit  Coverage 

Since  October  1984,  four  audit  reports  were  issued  that  discussed 
C-17  aircraft  issues  falling  within  the  scope  of  this  audit. 

-  Inspector  General,  DoD,  Audit  Report  No.  89-059, 
"Acquisition  of  the  C-17A  Aircraft,"  March  20,  1989.  This  audit 
reviewed  nine  elements  of  the  C-17  program  management  critical  to 
the  full-scale  development  phase  of  the  program.  It  identified 
the  on-time  development  of  properly  operating  software  as  the 
biggest  risk  to  the  program.  The  report  recommended  that  the  Air 
Force  require  Douglas  to  establish  an  acceptable  quality 
assurance  program  for  software  development.  To  establish  an 
adequate  software  quality  assurance  program,  the  Air  Force  and 
Douglas  agreed  to  a  tailored  version  of  Military  Specification 
52779A  (MIL-S-52779A)  and  made  it  part  of  the  FSD  contract.  The 
tailored  MIL-S-52779A  deleted  the  requirement  for  the  quality 
assurance  plan  to  provide  for  detecting,  reporting,  analyzing, 
and  correcting  software  problems  and  deficiencies.  At  the  same 
time,  the  tailored  version  of  MIL-S-52779A  retained 
paragraph  3.2.9,  which  requires  the  quality  assurance  plan  to 
reference  or  document  procedures  assuring  the  prompt  detection, 
documentation,  and  correction  of  software  problems  and 
deficiencies.  Based  on  the  results  of  the  current  audit,  we 
believe  that  procedures  referenced  or  documented  in  the  quality 
assurance  plan  should  be  the  procedures  actually  used  by  Douglas. 

Report  No.  89-059  also  recommended  that  the  Air  Force  improve 
guidance  on  the  performance  of  either  internal  independent 
verification  and  validation  (IIV&V)  or  independent  verification 
and  validation  (IV&V)  of  software.  The  Air  Force  nonconcur red 
with  improving  guidance  for  IIV&V  and  IV&V  stating  that  Air  Force 
Regulation  800-14,  "Life-cycle  Management  of  Computer  Resources 
in  Systems,"  September  29,  1986,  gives  program  directors  the 
latitude  to  tailor  program  requirements  based  on  the  program 
risk.  Changes  made  to  the  Aeronautical  Systems  Division  (ASD), 
Air  Force  Systems  Command,  policy  after  the  issuance  of  the 
report  have  lessened  the  concerns,  because  the  new  policy 
recognizes  that  using  IIV&V  for  all  contracts  would  be 
inappropriate.  The  principal  concern  was  that  ASD's  former 
policy  required  the  application  of  IIV&V  as  the  sole  means  to 
accomplish  IV&V.  In  deciding  how  to  accomplish  IV&V,  program 
directors  and  their  computer  resource  working  groups  should 
review  the  contractor's  prior  experience  and  the  Air  Force's 
capability  to  monitor  software  development  progress.  ASD's 
revised  policy,  to  allow,  but  not  to  mandate  IIV&V,  provides  the 
flexibility  needed  by  the  program  directors  to  select  the  most 
appropriate  method  of  IV&V. 
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The  report  further  recommended  that  the  Air  Force  fund  and  start 
an  IV&V  effort  for  the  C-17  program.  The  Air  Force  nonconcurred 
with  starting  an  IV&V  effort  stating  that  it  was  too  late  in  the 
program's  development  to  be  beneficial.  However,  in  1987,  the 
Air  Force  had  requested  Douglas  to  do  IIV&V  of  flight  critical 
software  at  no  additional  cost  to  the  Government.  In  1989, 
Douglas  tasked  the  Space  Systems  Company  of  the  McDonnell  Douglas 
Corporation  to  conduct  an  IIV&V  effort  for  the  flight  critical 
software  for  the  C-17.  The  Space  Systems  Company,  a  corporate 
component  independent  of  Douglas,  can  provide  an  independent  and 
effective  evaluation  of  the  flight  critical  software  for  the  C-17 
through  the  use  of  IIV&V.  Accordingly,  the  intent  of  the 

recommendation  in  Report  No.  89-059  is  considered  met. 

-  Inspector  General,  DoD,  Audit  Report  No.  89-067,  "Review  of 
the  C-17  Cargo  Aircraft  Program  as  a  Part  of  the  Audit  of  the 
Effectiveness  of  the  Defense  Acquisition  Board  Process,"  April  ~6T 
1989 .  This  audit  evaluated  the  C-17  program  compliance  with 
applicable  DoD  acquisition  directives  and  instructions.  It 
concluded  that  the  planned  Defense  Acquisition  Board  (DAB)  review 
in  October  1990  of  an  Air  Force  request  to  procure  10  C-17 

aircraft  in  FY  1991  and  20  C-17  aircraft  in  FY  1992  would 

represent  a  de  facto  full-rate  production  decision  normally  made 
at  a  Milestone  IIIB  review.  The  scope  of  the  planned 

October  1990  DAB  review  was  challenged  because  the  results  of  the 
operational  readiness  evaluation  and  initial  operational  test  and 

evaluation  would  not  be  known.  The  report  recommended  that  the 

scope  of  the  DAB  review  be  reduced  to  the  low-rate  initial 
production  quantity  of  10  aircraft  per  year  established  at  the 
Milestone  IIIA  (low-rate  initial  production)  review  held  on 
December  5,  1988.  OSD  nonconcurred  stating  that  the 

recommendation  was  judgmental  and  did  not  allow  for  improvements 
in  production  capability  obtained  from  increasing  production 
quantities.  In  response,  the  auditors  maintained  that  the  DAB 
should  exercise  caution  in  approving  large  procurement  quantities 
of  C-17  aircraft  before  operational  test  results  are  known. 

In  the  FY  1990  Appropriations  Act  for  DoD,  Congress  defined 
low-rate  initial  production  for  all  programs,  except  naval 
vessels  and  satellites,  as  the  minimum  quantity  necessary: 

(1)  to  provide  production-configured  or 
representative  articles  for  operational  tests 
pursuant  to  section  2399  of  this  title; 

(2)  to  establish  an  initial  production  base 
for  the  system; 

(3)  to  permit  an  orderly  increase  in  the 
production  rate  for  the  system  sufficient  to 
lead  to  full-rate  production  upon  the 
successful  completion  of  operational  testing. 
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Budget  reductions  made  by  the  Congress  because  of  development 
difficulties  reduced  C-17  procurement  quantities  for  FY's  1990, 
1991,  1992,  and  1993  from  6,  10,  20,  and  29  aircraft, 

respectively,  to  4,  6,  12,  and  24  aircraft,  respectively.  OSD 
officials  stated  that  the  revised  procurement  quantities 

represented  a  more  gradual  production  increase  and  were 
consistent  with  the  new  congressional  definition  of  low-rate 
initial  production.  The  Secretary  of  Defense's  decision  on 
April  26,  1990,  to  reduce  the  total  procurement  quantity  from 

210  C-17  aircraft  to  120  C-17  aircraft  will  further  reduce  the 

number  of  C-17's  to  be  purchased  each  year.  Accordingly,  the 

intent  of  the  recommendation  in  Report  No.  89-067  is  considered 
met. 


-  General  Accounting  Office  (GAOj  Report  No.  NSIAD-88-160 
(OSD  Case  No.  7590),  "DoD  Acquisition  Programs  -  Status  of 
Selected  Systems,*1  June  30,  1988.  GAO  reported  information  on 
requirements,  schedule,  performance,  cost,  and  funding  for  the 
C-17  aircraft  and  22  other  DoD  acquisition  programs.  GAO 
reported  that  Air  Force  and  contractor  officials  believed  that 
the  C-17  program  faced  technical  performance-related  challenges 
relating  to  hardware  and  software.  These  challenges  included 
maintaining  the  aircraft's  operating  weight  at  268,000  pounds  and 
integrating  the  avionics  into  the  aircraft  in  a  timely  manner. 
The  GAO  report  did  not  contain  any  recommendations.  We  discuss 
problems  in  developing  and  integrating  avionics  software  in 
Finding  B  of  this  report. 

-  GAO  Report  No.  NSIAD-89-195  (OSD  Case  No.  7992),  "C-17 
Faces  Schedule,  Cost,  and  Performance  Challenges,"  August  18, 
1989.  GAO  reported  that  the  C-17  Program  Director  considered  it 
unlikely  that  the  aircraft's  first  flight  date,  planned  at  that 
time  for  August  1990,  would  be  met.  The  program's,  capability  to 
meet  a  revised  first  flight  date  of  December  1990—/  was  dependent 
on  Douglas's  ability  to  resolve  assembly  and  avionics  development 
difficulties.  GAO's  report  discussed  delays  incurred  in  the 
development  of  the  mission  computer  system  and  the  electronic 
flight  control  system.  GAO  did  not  make  any  recommendations.  We 
discuss  similar  problems  in  meeting  development  milestones  and  in 
developing  avionics  software  in  Finding  B  of  this  report. 


17“  In  December  1989,  the  Air  Force  revised  the  first  flight  date 
to  June  1991. 
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PART  II  -  FINDINGS  AND  RECOMMENDATIONS 


A.  Mission  Computer  Software  Critical  Design  Review 

FINDING 


The  C-17  System  Program  Office  (SPO)  exercised  the  FY  1989 
contract  option  to  buy  four  C-17  aircraft  before  Douglas  Aircraft 
Company  (Douglas)  completed  the  required  critical  design  review 
(CDR)  of  all  mission  computer  software.  Completion  of  the  CDR,  a 
prerequisite  to  exercising  the  option,  was  required  by  a 
November  1988  modification  of  the  full-scale  development  ( FSD) 
contract.  The  C-17  Program  Director  considered  the  CDR,  made  in 
April  1989,  to  be  adequate  and  complete,  even  though  Douglas  had 
completed  the  detail  design  for  only  about  60  percent  of  all 
software  in  the  mission  computer.  The  C-17  program  development 
has  already  experienced  major  delays,  and  unless  all  required 
mission  computer  software  is  developed  before  the  C-17  passes 
from  the  developmental  test  and  evaluation  phase  to  the 
operational  test  and  evaluation  phase,  the  C-17  program  schedule 
will  be  further  delayed. 

DISCUSSION  OF  DETAILS 

Background.  The  FSD  contract  required  the  Air  Force  to 
exercise  the  FY  1989  production  option  (Lot  II)  for  up  to 
six  aircraft  on  or  before  January  31,  1989,  regardless  of  the 
program's  development  status,  or  the  Air  Force  would  have  to 
renegotiate  the  option  price.  Before  convening  the  Defense 
Acquisition  Board  (DAB)  in  1988,  OSD  officials  expressed  concern 
about  the  appropriateness  of  exercising  the  FY  1989  production 
option  when  major  delays  were  being  experienced  in  the  program's 
development.  In  response  to  this  concern,  the  C-17  Program 
Director  initiated  an  event-based  contract  award  strategy  for 
FY's  1989  through  1992.  The  Program  Director  stated  that  the 
purpose  of  this  strategy  was  to  encourage  Douglas  to  demonstrate 
confidence  in  its  capability  to  develop  the  C-17  aircraft  on 
schedule  and  to  give  the  Program  Director  added  flexibility  to 
influence  Douglas.  The  event-based  strategy  was  similar  to  a 
strategy  outlined  in  the  Air  Force  Systems  Command  Supplement  to 
the  Federal  Acquisition  Regulation,  subparagraph  17.270, 
"Demonstration  Milestones."  The  subparagraph  provides  that  for 
FSD  contracts  with  production  options,  the  option  exercise  period 
shall  be  tied  to  and  be  contingent  on  the  contractor's 
satisfactory  demonstration  of  its  ability  to  deliver  the  items  in 
the  option. 
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FSD  Contract  Modification.  In  November  1988,  the  Air 
Force  modified  the  FSD  contract  to  specify  certain  program  events 
that  had  to  take  place  annually  before  aircraft  would  be  procured 
in  FY's  1989  through  1992  (see  Appendix  A).  However,  to  give  the 
Government  flexibility,  the  contract  modification  also  provided 
that  the  Government  could  buy  the  aircraft  even  if  annual  program 
events  were  not  completed.  In  addition  to  establishing  program 
milestone  events,  the  FSD  contract  modification  stated  that  a 
production  contract  could  not  be  awarded  until  the  milestone 
event  for  FY  1990  was  accomplished.  Furthermore,  the  production 
contract,  when  awarded,  would  make  FY  1991  and  1992  procurements 
contingent  on  the  successful  completion  of  milestone  events 
described  in  the  FSD  contract  modification. 

The  Mission  Computer  CDR.  The  exercise  of  the 
procurement  option  for  FY  1989  was  made  contingent  on  the 
procuring  contracting  officer's  determination  that  the  mission 
computer  CDR  was  complete,  because  the  C-17  Program  Director 
believed  that  the  CDR  would  be  the  most  significant  measurable 
event  to  occur  in  FY  1989.  In  addition,  development  of  the 
mission  computer  is  crucial  to  complete  the  overall  avionics 
integration  and  to  keep  the  program  on  schedule.  Douglas  and  the 
Air  Force  did  a  CDR  of  the  C-17  aircraft  in  1988,  for  which 
Douglas  provided  the  mission  computer's  hardware  design,  but  not 
its  software  design.  During  the  CDR,  the  Air  Force  determined 
that  a  lack  of  progress  in  the  mission  computer  development  was  a 
serious  problem.  As  a  result,  the  SPO  withheld  $5.4  million  in 
payment  to  Douglas  pending  successful  completion  of  the  CDR  of 
mission  computer  software. 

The  FSD  contract  required  CDR's  of  the  subsystems  being  developed 
by  Douglas's  subcontractors  to  be  done  in  accordance  with  the 
intent  of  Military  Standard  1521A  (MIL-STD-1521A) ,  "Technical 
Reviews  and  Audits  for  Systems,  Equipments,  and  Computer 
Software."  The  Military  Standard  states  that  a  CDR  of  software 
shall  be  conducted  when  the  detail  design  is  essentially 
complete.  According  to  the  Military  Standard,  the  purpose  of  a 
CDR  of  software  is  to  determine  if  the  detail  design  of  software 
satisfies  performance  and  engineering  requirements.  The  primary 
product  of  the  CDR  is  the  identification  of  specific  computer 
programming  documentation  that  may  be  released  for  coding  and 
subsequent  testing.  CDR’s  may  be  tailored  to  the  level  of  review 
appropriate  to  the  weapon  system  and  its  subsystems. 
Accordingly,  a  simple  off-the-shelf  system  may  not  need  a  CDR, 
whereas  a  complex  system  comprised  of  an  aggregate  of  subsystems 
may  require  that  the  CDR  be  done  in  increments  as  the  subsystems 
are  developed. 

Because  the  mission  computer  for  the  C-17  aircraft  is  complex, 
its  CDR  was  being  done  in  increments.  The  mission  computer  acts 
as  the  heart  of  the  automated  avionics  system  and  performs 


8 


functions  normally  done  by  the  flight  engineer.  It  receives  data 
from  other  systems,  analyzes  data,  performs  calculations,  and 
displays  information  to  the  pilot  and  copilot.  Douglas  and  its 
subcontractor  for  the  mission  computer,  Delco  Systems  Operations 
(Delco),  General  Motors  Corporation,  made  incremental  reviews  of 
the  mission  computer  hardware  and  the  software  (three  reviews  of 
hardware,  one  review  of  software)  during  May  through  August  1988. 
Douglas  and  Delco  intended  for  the  review  of  the  mission  computer 
software  in  April  1989  to  be  the  final  increment  of  the  mission 
computer  CDR. 

Before  August  1988,  Delco  planned  to  complete  all  mission 
computer  software  before  the  C-17's  first  flight,  then  scheduled 
for  August  1990.  In  August  1988,  an  independent  review  team  that 
included  personnel  from  McDonnell  Douglas,  Douglas,  Hughes 
Electronics,  and  the  Air  Force  concluded  that  mission  computer 
software  requirements  were  not  well  defined,  but  that  not  all 
software  was  required  before  the  C-17's  first  flight.  Based  on 
the  results  of  the  independent  review  team,  the  mission  computer 
software  development  was  divided  into  two  new  segments.  The 
first  segment,  FT1 ,  consisted  of  software  needed  for  the  first 
100  hours  of  flight  testing.  The  second  segment,  FT2 ,  consisted 
of  all  FT1  and  residual  software  needed  for  the  aircraft  and 
would  be  used  for  testing  and  evaluating  the  avionics  system. 
FT1  and  FT2  were  interdependent;  any  change  to  FT1  would  affect 
the  design  and  coding  of  FT2.  Changes  to  FT2  need  not  affect 
FT1 ,  because  FT 2  included  some  independent  functions.  Despite 
the  change  in  the  software  development  strategy,  Douglas 
continued  to  plan  for  a  single  CDR  of  all  mission  computer 
software.  The  auditors  were  told  that  the  SPO  engineers  were 
concerned  that  the  FT2  software  would  not  be  ready  for  the  1989 
CDR,  and  that  the  SPO  engineers  wanted  Douglas  to  perform  a 
review  of  all  the  software  after  the  CDR  was  performed.  However, 
we  found  no  evidence  before  the  FSD  contract  was  modified  in 
November  1988  to  incorporate  the  event-based  strategy  that 
Douglas  agreed  to  do  a  software  review  after  the  CDR,  or  that  the 
time  and  scope  of  that  software  review  was  decided. 

Scope  of  the  Mission  Computer  CDR.  In  our  opinion,  the  CDR 
of  the  mission  computer  software  was  a  significant  event; 
however,  it  demonstrated  that  a  well-defined  transition  point  was 
not  attained  between  detail  design  and  coding  and  testing  of  all 
mission  computer  software.  The  CDR  was  important  to  SPO 
officials,  because  it  was  the  first  time  that  the  Air  Force  could 
see  how  Douglas  planned  to  implement  the  mission  computer 
functions.  According  to  the  CDR  plan  prepared  by  Douglas  in 
February  1989,  the  objective  of  the  CDR  was  to  review  Delco' s 
detail  design  solutions  to  show  that  Delco 's  design  would  satisfy 
contractual  requirements.  According  to  the  plan,  CDR  tasks 
included  discussion  of  performance  and  design  requirements,  a 
thorough  review  of  the  software  design  concept  and  detail  design 
and  an  evaluation  of  overall  development  status.  To  perform  the 
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CDR,  the  plan  required  that  95  percent  of  the  software 
requirements  be  defined/  80  percent  of  the  overall  top-level 
software  design  be  completed,  and  90  percent  of  the  detail  design 
of  the  software  needed  for  FT1  be  completed.  Only  60  percent  of 
the  detail  design  for  software  for  the  mission  computer  was 
needed  for  FT1.  Although  the  CDR  plan  also  called  for  the 
completion  of  50  percent  of  all  detail  design  for  software  before 
the  CDR,  this  criterion  could  be  met  without  any  FT 2  detail 
design.  Before  performing  the  CDR,  the  SPO  chief  avionics 
engineer  determined  that  although  mission  computer  software 
requirements  and  design  were  done  in  accordance  with  provisions 
in  the  CDR  plan,  they  were  not  sufficiently  complete.  Because 
only  the  detail  design  of  the  FT1  software  was  to  be  essentially 
completed  for  the  CDR,  the  chief  avionics  engineer  concluded  that 
the  mission  computer  represented  a  high  risk  (likely  to  affect 
performance,  schedule,  or  cost  even  with  contractor  emphasis  and 
close  Government  monitoring). 

Performance  of  the  Mission  Computer  CDR.  Although  prerequi¬ 
sites  to  performing  the  CDR  were  technically  met,  the  CDR  did  not 
include  a  review  of  the  detail  design  of  software  for  FT1  or  FT2 
as  required  by  the  CDR  plan.  From  April  10  to  April  14,  1989, 
Douglas  performed  the  CDR  of  the  mission  computer  software  being 
developed  by  Delco.  The  Air  Force's  participation  was  limited  to 
asking  questions,  because  the  FSD  contract  makes  Douglas  totally 
responsible  for  developing  the  aircraft.  Two  Air  Force 
participants  in  the  CDR  told  the  auditors  that  overall  software 
requirements  and  FT1  and  FT 2  top-level  designs  for  the  mission 
computer  were  discussed  during  the  CDR,  but  no  detail  designs  for 
software  were  discussed  in-depth.  According  to  the 
two  participants,  Delco  officials  turned  aside  questions  about 
the  FT1  detail  design  saying  the  design  was  available  for  review 
in  a  library  of  software.  During  the  audit,  Delco  officials  told 
the  auditors  that  the  FT 2  detail  design,  which  was  only  8  percent 
complete  at  the  time  of  the  CDR,  was  not  presented.  The  auditors 
were  told  by  a  SPO  official  that  because  of  time  limitations,  the 
principal  objective  of  the  CDR  was  to  determine  if  Delco 
understood  system  requirements  and  not  to  determine  if  Delco  had 
completed  the  detail  design  of  FT1.  The  SPO  official  stated  that 
the  top-level  design  and  the  detail  design  of  a  few  FT1  software 
modules  were  reviewed  to  check  Delco' s  understanding. 

We  audited  Air  Force,  Douglas,  and  Delco  actions  taken  to  correct 
nine  review  items  written  by  the  Air  Force  and  Douglas  during  the 
mission  computer  software  CDR.  Review  items  are  critical 
deficiencies.  The  C-17  Program  Director  required  that  the  review 
items  be  corrected  or  that  corrective  action  plans  be  developed 
before  the  FY  1989  production  option  would  be  exercised.  We 
determined  that  the  corrective  action  plans  as  briefed  by  Douglas 
to  the  Air  Force  on  June  5,  1989,  were  adequate.  We  believe  that 
the  corrective  action  plans  provided  a  sufficient  basis  to 
consider  the  review  items  corrected,  because  the  corrective  work 
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required  several  months  to  complete  and  posed  no  obstacle  to 
development  of  the  mission  computer  if  not  corrected  immediately. 
The  review  items  and  the  status  of  the  corrective  actions  as  of 
February  2,  1990,  are  summarized  in  Appendix  B. 

Completion  of  Mission  Computer  Software.  The  performance  of 
the  mission  computer  software  CDR  in  April  1989  was  instrumental 
in  Douglas  and  Delco  developing  a  plan  to  complete  the  mission 
computer  and  in  resolving  management  difficulties  between  Douglas 
and  Delco.  During  the  CDR,  the  Air  Force  requested  a  plan  for 
completing  the  mission  computer  software,  but  Delco  did  not 
provide  a  plan.  The  last  review  item  (Air  Force-065)  was  written 
by  the  Air  Force  to  force  Delco  to  prepare  a  plan  for  completing 
the  mission  computer  software.  Ultimately,  Douglas  presented  a 
plan  to  the  SPO  on  June  5,  1989.  The  plan  proposed  a  software 
design  review  in  October  1990  — '  to  review  the  detail  design  of 
FT2 .  Although  scheduled  as  a  software  design  review,  SPO 
officials  stressed  that  the  software  design  review  would  not  be 
reduced  in  importance  from  the  CDR.  The  SPO  officials  added  that 
the  software  design  review  was  not  called  a  CDR  because  the  CDR 
occurred  in  April  1989,  and  the  October  1990  review  required  less 
preparation  by  the  participants.  The  SPO  officials  stated  that, 
except  in  name,  the  software  design  review  will  constitute  a  CDR 
of  FT 2  software. 

Before  exercising  the  1989  option,  the  C-17  Program  Director 
wanted  Douglas  and  Delco  to  agree  on  their  respective 
responsibilities  in  completing  the  mission  computer  work  and  to 
commit  to  meeting  the  program  schedule.  Douglas  and  Delco 
officials  told  us  that  during  the  CDR,  Delco  was  unwilling  to 
commit  to  a  plan  for  completing  the  mission  computer  software  due 
to  a  poor  working  relationship  with  Douglas.  The  changes  clause 
in  the  contract  between  Delco  and  Douglas  provided  that  Douglas 
could  make  changes  within  the  general  scope  of  the  contract 
without  compensating  Delco.  Delco  believed  that  Douglas  was 
initiating  software  requirements  changes  that  were  unreasonable 
and  that  were  outside  the  contract  scope.  Moreover,  Delco 
officials  told  us  that  the  expense  of  making  the  changes  was 
becoming  onerous. 

On  July  18,  1989,  Douglas  and  Delco  signed  a  memorandum  of 
agreement  that  curtailed  Delco' s  role  in  developing  the  mission 
computer  software.  Douglas  terminated  Delco 's  responsibility  for 
mission  computer  software  system  engineering,  development,  and 
integration  and  testing,  but  retained  Delco  personnel  to  continue 
working  on  the  C-17  project.  Delco  remained  responsible  for 
developing  mission  computer  hardware  and  software  test  equipment. 
On  July  24,  1989,  Douglas's  vice  president-general  manager  for 


— /  We  were informed  on  October  22,  1990,  that  the  software  design 
review  had  been  rescheduled  to  December  1990. 


11 


the  C-17  wrote  to  the  C-17  Program  Director  stating  that  the 
mission  computer  CDR  could  be  considered  completed  based  on  the 
development  of  a  plan  for  completing  the  mission  computer 
software  and  on  Douglas's  assumption  of  management  responsibility 
from  Delco  for  the  mission  computer  software.  Delco's  general 
director  advised  Douglas  in  a  letter  dated  July  27,  1989,  that 
Delco's  corporate  parent,  Delco  Electronics  Corporation, 
considered  the  memorandum  of  agreement  with  Douglas  to  be 
acceptable.  The  C-17  Program  Director  believed  that  this 
commitment  from  Delco  and  Delco  Electronics  Corporation  provided 
adequate  assurance  that  measures  were  in  place  to  complete 
development  of  the  C-17  mission  computer  software. 

The  C-17  Program  Director  considered  the  CDR  to  be  completed 
based  on  the  review  of  the  top-level  software  design  in 
April  1989  and  on  management  actions  taken  by  Douglas.  He  did 
not  consider  the  "software  design  review"  then  scheduled  for 
October  1990  to  be  part  of  the  CDR  or  to  have  any  bearing  on  the 
event-based  contract  award  strategy.  The  SPO  exercised  the 
contract  option  to  buy  four  C-17  aircraft  on  July  28,  1989. 
Additional  problems  in  developing  mission  computer  software  are 
discussed  in  Finding  B  of  this  report. 

We  believe  that  the  April  1989  CDR  represented  an  excellent 
opportunity  for  the  SPO  and  Douglas  to  assess  the  mission 
computer  software  development  status  and  to  assess  Delco |s 
understanding  of  the  mission  computer  requirements.  However,  in 
our  opinion,  the  CDR  was  not  complete  because  it  did  not  include 
a  review  of  the  detail  design  of  the  mission  computer  software. 
Although  MIL-STD-1 5 21A  allows  the  criteria  for  conducting  CDR's 
to  be  tailored  to  meet  specific  program  needs,  the  criteria 
should  be  rigorous  when  the  software  development  task  is  complex 
and  the  software  developers  lack  experience,  as  was  the  case  with 
the  C-17  mission  computer.  We  believe  that  to  perform  the  CDR  in 
April  1989  was  premature  because  of  the  interdependency  of  FT1 
and  FT  2  software,  because  the  SPO  and  Douglas  knew  that 
significant  mission  computer  software  development  problems 
existed,  and  because  only  60  percent  of  the  overall  detail  design 
for  software  was  complete.  In  our  opinion,  the  April  1989  CDR 
only  when  taken  together  with  the  software  design  review  to  be 
done  in  late  1990  constitutes  a  completed  CDR  of  the  mission 
computer  software.  Although  the  C-17  Program  Director  had  the 
prerogative  to  exercise  the  contract  option  at  any  time, 
notwithstanding  CDR  results,  we  believe  that  he  should  not  have 
considered  the  CDR  to  be  completed. 

Future  Milestone  Event.  The  milestone  event  for  FY  1990  in 
the  FSD  contract  needs  to  be  better  defined  by  the  C-17  Program 
Director  to  eliminate  possible  future  misunderstandings.  The 
milestone  event  makes  procurement  of  up  to  six  aircraft  in  1990 
contingent  on  final  assembly  of  the  T-l  (test  aircraft), 
scheduled  to  be  completed  in  December  1990.  At  the  end  of  our 
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audit,  Douglas  and  the  Air  Force  were  attempting  to  agree  on  the 
meaning  of  "T-l  final  assembly,"  and  on  which  components  could  be 
missing  and  still  have  the  test  aircraft  be  considered  assembled. 

OSD  Oversight.  Members  of  the  DAB  Conventional  Systems 
Committee  (the  Committee)  were  unaware  that  the  CDR  of 
C-17  mission  computer  software  done  in  April  1989  was  not  a 
comprehensive  review  of  the  detail  design  of  mission  computer 
software.  In  preparation  for  the  Milestone  IIIA  (low-rate 
initial  production)  review  in  December  1988,  the  C-17  Program 
Director  had  informed  the  Committee  that  the  software  was  being 
developed  in  increments,  and  that  the  first  increment  included 
software  required  for  the  first  100  hours  of  flight  testing.  The 
C-17  Program  Director's  program  status  briefing  to  the  Committee 
in  November  1988  indicated  that  the  CDR  would  be  completed  in 
March  1989  and  provided  the  status  of  the  software  development.  A 
briefing  chart  showed  that  as  of  November  10,  1988,  55  percent  of 
the  top-level  design  and  36  percent  of  the  overall  detail  design 
was  completed.  OSD  officials  informed  the  auditors  that  no 
Committee  members  had  expertise  in  computer  software 
developments.  If  experts  in  software  developments  had  been 
present  during  the  program  status  briefing,  we  believe  the 
Committee  would  have  questioned  more  rigorously  whether  Douglas 
would  be  able  to  complete  all  detail  design  by  the  scheduled  CDR, 
or  would  have  advised  against  doing  the  CDR  with  so  much  of  the 
total  detail  design  yet  undone.  An  OSD  official  indicated  that 
although  the  Committee  still  might  have  recommended  approval  of 
the  FY  1989  procurement,  if  members  of  the  Committee  had 
expertise  in  software  developments,  it  might  have  recommended 
that  completion  of  a  CDR  of  all  mission  computer  software  be  a 
prerequisite  to  the  FY  1990  buy.  At  the  Milestone  IIIA  review, 
the  DAB  approved  C-17  procurements  for  FY's  1989  and  1990  and  an 
advance  procurement  in  FY  1990  for  FY  1991  production  contingent 
on  only  the  attainment  of  each  program  milestone  event  stated  in 
the  FSD  contract. 

The  Office  of  the  Deputy  Director  of  Defense  Research  and 
Engineering  (Research  and  Advanced  Technology),  Office  of  the 
Under  Secretary  of  Defense  for  Acquisition,  is  responsible  for 
many  functions,  one  of  which  is  software  development  policy  for 
mission  critical  computer  systems.  With  only  one  seat  on  the 
Committee,  this  office  is  unable  to  send  individuals  with 
expertise  in  software  and  in  all  other  system  development  areas 
to  the  Committee  meetings.  Furthermore,  software  development 
experts  in  Research  and  Advanced  Technology  perform  other 
functions  that  detract  from  the  time  needed  for  weapon  system 
reviews.  We  believe  that  sufficient  personnel  having  expertise 
in  software  development  matters  should  be  present  at  Committee 
meetings  to  provide  effective  oversight  of  software  developments. 

Following  our  audit,  we  learned  that,  as  part  of  the 
reorganization  of  the  acquisition  management  structure,  OSD  had 
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proposed  steps  to  improve  the  oversight  of  software  development. 
On  February  9,  1990,  the  Deputy  Director  of  Defense  Research  and 
Engineering  (Research  and  Advanced  Technology)  published  for 
review  and  comment  a  preliminary  draft  of  a  DoD  Software  Master 
Plan.  The  draft  master  plan  identifies  27  actions  to  meet 
challenges  facing  DoD  that  relate  to  software  development, 
utilization,  and  cost.  Included  is  a  proposal  to  establish 
single  offices  at  OSD  and  the  Military  Department  levels  for 
managing  and  implementing  software  acquisition  and  life-cycle 
management  policies.  The  plan  also  includes  a  proposal  to  revise 
policy  directives  and  instructions  to  ensure  that  software 
development  matters  are  addressed  by  the  DAB.  We  believe  that 
establishing  these  offices  and  revising  the  policy  directives  and 
instructions  will  help  ensure  that  software  developments  are 
properly  addressed  by  the  DAB  and  its  subordinate  committees. 

Conclusion.  We  support  the  Air  Force's  use  of  the 
event-based  contract  award  strategy  for  the  C-17  aircraft. 
However,  we  believe  that  milestone  events  used  in  the  strategy 
should  contain  definitive  criteria  to  exercise  contract  options 
to  ensure  that  the  Government  and  the  contractor  fully  understand 
contractual  requirements.  Personnel  with  technical  expertise 
providing  oversight  in  software  developments  would  help  ensure 
that  acquisition  programs  heavily  reliant  on  software  receive  the 
appropriate  scrutiny  before  procurement  decisions  are  made. 

RECOMMENDATIONS  FOR  CORRECTIVE  ACTION 

1.  We  recommend  that  the  Under  Secretary  of  Defense  for 
Acquisition  require  that  individuals  with  appropriate  technical 
expertise  attend  meetings  of  the  Conventional  Systems  Committee 
of  the  Defense  Acquisition  Board  when  their  oversight  of  software 
development  matters  is  needed. 

2.  We  recommend  that  the  C-17  Program  Director,  Air  Force 
Systems  Command,  amend  the  C-17  full-scale  development  contract 
with  Douglas  Aircraft  Company  to: 

a.  Require  completion  of  the  software  design  review  of  the 
mission  computer  software  as  a  precondition  to  awarding  the  Lot 
III  production  contract  to  procure  four  aircraft  for  FY  1990. 

b.  Specify  the  minimum  assembly  required  to  consider  the 
test  aircraft  (T-l)  assembled  for  the  milestone  event  for 
Lot  III. 

MANAGEMENT  COMMENTS 


The  Director  of  Defense  Research  and  Engineering  nonconcurred 
with  Recommendation  A.I.,  because  principals  of  the  Committee  are 
allowed  to  bring  supporting  staff  members  with  relevant  expertise 
to  meetings.  The  Director  agreed  with  the  intent  of  the 
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recommendation  to  strengthen  oversight  of  software  development, 
and  agreed  that  actions  proposed  in  the  preliminary  draft  of  the 
DoD  Software  Master  Plan  would  strengthen  software  oversight.  A 
complete  text  of  the  comments  is  in  Appendix  C. 

The  Air  Force's  Deputy  Assistant  Secretary  (Acquisition) 
nonconcur red  with  the  finding  and  the  recommendations.  He  stated 
that  Douglas  met  all  requirements  of  the  CDR  and  that  the  Air 
Force  and  Douglas  considered  the  CDR  to  be  completed.  He 
indicated  that  the  auditors  used  an  academic  definition  of  a  CDR 
rather  than  what  had  been  agreed  to  by  the  Air  Force  and 
Douglas.  Although  he  nonconcurred  with  Recommendation  A. 2. a.,  he 
stated  that  the  "SPO  and  DAC  [Douglas  Aircraft  Company]  have 
reached  an  agreement  on  a  more  precise  definition  of  the  T-l 
assembly  complete  milestone."  Complete  comments  are  provided  in 
Appendix  D. 


AUDIT  RESPONSE  TO  MANAGEMENT  COMMENTS 

Regarding  Recommendation  A.I.,  the  Committee's  policy  of  allowing 
principal  staff  members  to  bring  individuals  with  the  relevant 
technical  expertise  to  Committee  meetings  could  provide  proper 
oversight,  provided  the  experts  do  attend  the  meetings.  However, 
individuals  with  technical  expertise  in  software  matters  did  not 
attend  Committee  meetings  on  the  C-17  prior  to  the  DAB  review  in 
December  1988,  despite  the  fact  that  the  C-17  Program  Director 
had  identified  software  as  one  of  his  primary  concerns  with  the 
program.  We  believe  that  proper  OSD  oversight  of  software 
developments  on  major  weapon  systems  in  general  and  the  C-17 
avionics  system  in  particular  demands  that  software  experts 
regularly  participate  in  DAB  milestone  decisions.  We  agree  with 
the  Director  that  principal  members  can  bring  staff  with  relevant 
expertise  to  Committee  meetings  when  their  oversight  is  needed 
and  we  have  revised  the  recommendation  accordingly.  Therefore, 
we  ask  that  management  provide  comments  on  revised  Recommendation 
A. 1 .  in  response  to  the  final  report. 

Regarding  Recommendation  A. 2. a.,  the  auditors  evaluated  the  CDR 
in  accordance  with  the  definition  contained  in  MIL-STD-1521A.  We 
could  find  no  evidence  that  the  Air  Force  and  Douglas  had  agreed 
to  anything  less  than  a  complete  CDR  before  the  contract's 
modification  in  November  1988.  The  contractual  language  of 
Special  Provision  H-lll  (see  Appendix  A)  does  not  indicate  that 
less  than  a  complete  CDR  was  contemplated  or  agreed  on.  The  CDR 
plan,  developed  in  February  1989,  provided  criteria  for  holding 
the  CDR.  As  discussed  on  page  9,  the  criteria  allowed  the  CDR  to 
occur  before  the  detail  software  design  of  the  total  mission 
computer  was  essentially  completed.  We  maintain  that  the  Program 
Director  should  not  have  considered  the  CDR  to  be  completed, 
because  only  about  60  percent  of  the  detail  design  was 
completed.  However,  we  accept  that  both  the  Air  Force  and 
Douglas  consider  the  CDR  to  be  completed.  Because  the  mission 
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computer  is  critical  for  full  operational  performance  of  the  C-17 
aircraft,  we  believe  it  is  important  that  the  Air  Force  not 
commit  to  procuring  more  C-17  aircraft  until  the  software  design 
review  of  FT2  is  completed.  We  ask  that  management  reconsider 
its  position  on  Recommendation  A. 2. a.  in  response  to  the  final 
report . 

Although  the  Deputy  Assistant  Secretary  (Acquisition) 
nonconcurred  in  Recommendation  A.2.b.,  the  actions  taken  by  the 
Air  Force  to  obtain  a  definition  of  T-l  assembly  meet  the  intent 
of  the  recommendation.  Accordingly,  additional  management 
comments  on  Recommendation  A.2.b.  are  not  required. 
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B.  Engineering  Planning  and  Subcontractor  Management 

FINDING 


Douglas  had  not  fully  accomplished  technical  requirements  to  meet 
contractual  milestones  for  the  development  of  the  C-17  aircraft 
and  had  not  prepared  an  adequate  integration  test  plan  for  the 
development  and  integration  of  software  for  the  avionics  system. 
The  total  performance  responsibility  contract  that  the  Air  Force 
has  with  Douglas  does  not  specify  all  management  plans  necessary 
for  the  successful  development  of  the  C-17  program  and  restricts 
the  actions  of  the  SPO  to  ensure  that  milestones  are 
accomplished.  Delays  in  the  development  of  the  aircraft  have 
caused  the  Air  Force  to  postpone  major  program  milestones/  and 
further  delays  may  occur. 

DISCUSSION  OF  DETAILS 

Background.  The  C-17  full-scale  development  ( FSD)  contract, 
F33657-81-C-2108,  places  total  system  development  responsibility 
on  Douglas.  Special  Provision  H-21  states: 

The  Contractor  hereby  accepts  Total  System 
Responsibility  for  the  C-X  Transport 
Aircraft  whether  or  not  such  systems  (or 
subsystem,  components  thereof,  or  CFP 
[contractor-furnished  property])  are 
fabricated,  manufactured,  or  assembled  by 
the  prime  contractor,  subcontractor  .  .  . 
or  furnished  as  GFP  [Government-furnished 
property] . 

The  C-17  contract  Special  Provision  H-9,  "Subcontractor 
Management,"  holds  Douglas  responsible  to  effectively  manage 
subcontractors  and  to  " .  .  .  apply  special  management  emphasis  in 
surveillance  of  major/critical  subcontractor's  performance  to 
provide  reasonable  assurance  that  contractual  requirements  will 
be  met."  The  C-17  contract  lists  critical  program  milestones  in 
Special  Provision  H-ll,  "Demonstration  Milestones,"  and  Special 
Provision  H-lll,  "Prerequisite  to  Outyear  Production  Award." 
Special  Provision  H-ll  establishes  contractual  milestones  for 
developing  the  aircraft.  Special  Provision  H-lll  stipulates 
events  that  must  be  accomplished  before  procuring  aircraft  during 
FY's  1989  through  1992  and  is  discussed  further  in  Finding  A. 

The  Government  has  no  legal  basis  to  direct  the  actions  of  a 
subcontractor  except  through  the  prime  contractor.  This 

prohibition  is  referred  to  as  the  no  privity  rule.  Privity  of 
contract  means  that  relationships  exist  only  between  the 
Government  and  the  prime  contractor,  between  the  prime  contractor 
and  the  first  tier  subcontractor,  and  so  on. 
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Performance  Specifications.  A  specification  is  a  description 
of  the  technical  requirements  for  a  material,  product,  or 
service,  including  the  procedure  by  which  it  will  be  determined 
that  the  requirements  have  been  met.  A  detail  design 
specification  tells  the  contractor  exactly  what  to  build  and  how 
to  build  it.  For  developmental  programs  such  as  the  C-17,  the 
detail  design  specification  is  not  established  until  the 
functional  and  physical  configuration  audits  are  completed.  A 
performance  specification  tells  the  contractor  the  performance 
characteristics  that  the  system  must  have  in  order  to  be  accepted 
by  the  Government.  With  the  exception  of  certain  military  design 
standards  that  must  be  observed,  the  prime  contractor  is  free  to 
devise  any  design  that  will  meet  these  performance 
specifications.  Performance  specifications  allow  contractors  to 
use  existing  commercial  products  instead  of  developing  new  items. 

Performance  specifications  are  good  for  the  Government  because 
they  place  responsibility  for  designing  and  developing  the  system 
on  the  contractor  who  has  the  necessary  expertise,  resources,  and 
profit-oriented  motivation.  However,  the  prime  contractor  must 
fully  understand  the  system  requirements,  be  technically  capable 
of  developing  the  system,  and  have  management  controls  in  place 
to  ensure  that  performance  requirements  can  be  met.  To  help 
ensure  that  the  contractor  is  competent  and  properly  organized. 
Air  Force  Systems  Command  (AFSC)  Regulation  800-21, 
"Subcontracting  Management,"  July  16,  1987,  requires  the  SPO  to 
communicate  with  the  prime  contractor  and  to  evaluate  the  prime 
contractor's  subcontracting  and  program  management  to  ensure  that 
management  "operates  in  a  preventive  rather  than  reactive  mode." 

Program  Technical  Requirements.  Contractor  and  SPO 
officials  told  the  auditors  that  Douglas  and  the  Air  Force 
initially  underestimated  the  difficulty  of  developing  the  C-17 
aircraft  because  they  assumed  that  the  program  had  low  technical 
risk  and  low  schedule  risk.  Douglas  and  the  Air  Force  believed 
that  the  C-17  development  program  had  low  technical  risk  because 
the  design  of  the  C-17  relied  heavily  on  demonstrated 
technologies.  Although  no  new  technologies  have  been  developed 
for  the  C-17,  the  aircraft  is  a  new  cargo  air  lifter  that  is 
dependent  on  a  complex  integrated  avionics  system  to  reduce  the 
aircrew  size  to  two  pilots  and  one  cargo  loadmaster.  In  our 
opinion,  Douglas  did  not  recognize  the  schedule  risk  because  the 
C-17  was  Douglas's  first  attempt  at  developing  a  modern  military 
cargo  aircraft,  and  because  Douglas  lacked  experience  in 
integrating  complex  avionics  systems.  Military  cargo  aircraft 
must  be  capable  of  airdropping  cargo  and  personnel,  operating 
into  and  out  of  austere  airfields,  and  performing  other  missions 
not  required  of  commercial  aircraft. 

Douglas  selected  subcontractors  with  the  specialized  skills  to 
develop  64  subsystems  for  the  avionics  system  and  gave  its 
subcontractors  general  system  requirements  for  developing  the 
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subsystems.  Subcontracting  the  complete  development  of 
subsystems  is  Douglas's  normal  business  practice,  and  it  is  an 
effective  strategy  when  requirements  are  straightforward. 
However,  the  lack  of  detailed  requirements  and  timely  guidance 
caused  subcontractors  to  experience  problems  in  developing  the 
subsystems . 

Electronic  Flight  Control  System.  On  June  12,  1986, 
Douglas  awarded  I  firm-fixed-price  contract  to  Sperry 
Corporation,  which  was  later  purchased  by  Honeywell,  Inc.,  to 
develop  a  flight  control  system  for  the  C-17.  The  proposed 
design  was  for  a  dual-channel,  hybrid  fly-by-wire  and 
hydromechanical  flight  control  system.  In  a  fly-by-wire  system, 
hydraulic  actuation  of  aircraft  control  surfaces  (e.g.,  slats, 
flaps,  rudders,  and  stabilizers)  is  controlled  electronically 
rather  than  mechanically.  In  May  1987,  Douglas  directed 
Honeywell  to  change  the  flight  control  system  to  a  quadruple- 
redundant,  digital,  fly-by-wire  system,  with  mechanical  backup. 
Douglas  directed  this  change  after  confirming  that  in  wind  tunnel 
tests,  the  aircraft  configuration  and  original  flight  control 
system  could  allow  the  pilot  to  put  the  aircraft  into  an 
uncontrollable  stall  during  certain  tactical  maneuvers.  The 
potential  for  a  stall  was  initially  identified  in  1985  during  a 
preliminary  design  review  for  the  overall  aircraft  design.  The 
change  in  flight  control  systems  forced  Honeywell  to  redefine  the 
subsystem  requirements;  however,  the  program  development  schedule 
was  not  changed.  In  order  to  fully  define  the  subsystem 
requirements  and  to  be  able  to  design  the  flight  control  hardware 
and  software,  Honeywell  first  had  to  determine  how  the  electronic 
flight  control  system  (EFCS)  interacted  with  subsystems  being 
developed  by  other  subcontractors  and  to  prepare  an  interface 
control  document  (ICD)  defining  the  software  interactions  between 
subsystems.  Honeywell  did  not  complete  the  ICD  until  July  1989. 

During  1988  and  1989,  Douglas  grew  increasingly  concerned  about 
Honeywell  missing  schedule  milestones.  In  November  1988, 
Honeywell  identified  a  7-month  schedule  delay  for  the 
first  software  engineering  development  units.  In  March  1989, 
Honeywell  and  Douglas  signed  a  memorandum  of  agreement  (MOA)  that 
identified  a  further  delay  of  4  months  in  the  development 
schedule.  In  April  1989,  an  Air  Force  review  team  determined 
that  stronger  communications  were  a  must  between  Douglas  and 
Honeywell  on  system  and  interface  issues.  On  June  6,  1989, 
Honeywell  identified  an  additional  7-month  delay  from  the 
delivery  date  in  the  March  1989  MOA,  and  established  April  25, 
1991,  as  the  new  date  for  delivering  flight  qualified  software. 
On  June  16,  1989,  Douglas  and  Aircraft  Electronics  Division, 
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General  Electric  (GE),  signed  an  MOA  for  GE  to  develop  an  EFCS 
concurrently  with  Honeywell.  On  July  21 ,  1989 ,  Douglas  ended 
Honeywell's  contract  for  the  EFCS. 

To  develop  the  EFCS,  GE  was  using  its  previously  developed 
hardware  and  software  and  the  ICD  developed  by  Honeywell.  GE  had 
developed  similar  systems  for  the  B-2,  F-15E,  and  F-18  aircraft. 
GE  was  developing  the  EFCS  in  two  increments  and  was  working 
ahead  of  its  schedule  to  deliver  the  first  increment  by 
October  31,  1990,  when  it  is  needed  for  avionics  integration. 
Although  GE  had  committed  resources  to  the  program  and  was 
developing  the  system  for  the  C-17,  at  the  end  of  our  audit, 
Douglas  still  had  not  finalized  the  terms  and  conditions  of  GE's 
contract.  We  were  told  by  Douglas  and  GE  officials  that  contract 
negotiations  were  in  process. 

Mission  Computer.  Douglas  awarded  a  firm-fixed-price 
contract  to  Delco  on  July  10,  1986,  to  develop  the  mission 
computer.  In  August  1988,  an  independent  review  team  that 
included  personnel  from  McDonnell  Douglas,  Douglas,  Hughes 
Electronics,  and  the  Air  Force  concluded  that  system  engineering 
had  been  inadequately  accomplished  and  that  the  mission  computer 
system  requirements  were  not  adequately  defined.  During  late 
1988  and  early  1989,  Douglas  and  Delco  tried  to  establish  the 
baseline  requirements.  Although  development  of  the  baseline 
requirements  served  as  a  means  to  force  Douglas  and  Delco  to 
discuss  requirements,  Delco  officials  stated  that  documented 
baseline  requirements  were  not  formalized  and  included  in  the 
contract.  Delco  sufficiently  developed  the  mission  computer 
software  to  hold  a  CDR  of  the  detail  design  in  April  1989  for  the 
first  of  two  increments  of  software.  Delco,  however,  would  not 
commit  to  a  plan  for  completing  the  mission  computer,  because  the 
mission  computer  requirements  and  Delco 's  responsibilities  had 
not  been  adequately  defined.  On  July  18,  1989,  Douglas  and  Delco 
signed  an  MOA  that  partially  terminated  Delco 's  contract  for  the 
mission  computer  subsystem.  Douglas  assumed  the  responsibility 
to  manage  the  overall  software  development  effort  for  the  mission 
computer.  Delco  remained  responsible  for  the  hardware 
development  portion  of  the  contract.  For  software  development, 
Delco  is  providing  personnel  and  equipment  on  a  time  and 
materials  basis.  On  November  30,  1989,  Douglas  gave  Delco  a 
letter  contract  formalizing  the  changes  in  responsibilities. 

Douglas's  actions  kept  the  software  development  program  from 
worsening,  but  the  actions  clearly  indicated  that  software 
development  for  the  C-17  program  was  in  serious  trouble  late  in 
the  program.  Douglas's  management  actions  may  speed  up  the 
development  of  the  software  for  the  EFCS  and  mission  computer, 
but  Douglas  still  faces  integrating  all  the  subsystems  into  a 
functional  avionics  system. 
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Management  Plans  and  Organization.  Douglas  has  not  prepared 
the  engineering  plans  necessary  to  ensure  that  the  C-17  will  be 
developed  on  time.  In  addition,  Douglas  did  not  effectively 
staff  the  C-17  development  with  qualified  people  to  manage  the 
program.  Two  key  planning  tools  that  can  be  used  to  ensure  that 
contractors  have  properly  defined  requirements  and  set  procedures 
in  place  to  manage  major  acquisition  programs  are  the  systems 
engineering  management  plan  (SEMP),  and  the  avionics  integration 
test  plan.  The  SPO  did  not  require  Douglas  to  develop  a  SEMP  or 
its  equivalent.  We  discussed  the  importance  of  a  SEMP  with 
officials  at  another  Air  Force  SPO  that  was  responsible  for 
developing  a  fighter  aircraft.  That  SPO  told  us  that  the  SEMP 
was  important  to  ensure  that  the  contractor  has  adequately 
planned  the  development  effort.  In  fact,  the  SPO  for  that 
fighter  aircraft  had  developed  a  SEMP  for  its  prime  contractor. 

At  the  end  of  our  audit,  Douglas  still  had  not  prepared  an 
adequate  avionics  integration  test  plan  and  had  not  adequately 
staffed  its  management  of  the  C-17  program  to  perform  the  test 
and  integration  of  the  avionics  system.  For  reasons  discussed  in 
the  following  paragraphs,  we  believe  that  the  lack  of  an 
effective  avionics  integration  test  plan  and  inadequate  staffing 
for  integration  testing  pose  twin  risks  to  the  program  schedule, 
especially  because  the  C-17  program  is  Douglas's  first 
significant  effort  at  integrating  a  complex  avionics  system. 

Systems  Engineering  Management  Plan.  When  Military 
Standard  499A  (MIL-STD-499A) ,  "Engineering  Management,"  is 
included  in  the  contract,  a  SEMP  is  required  to  be  submitted 
within  the  contractor's  proposal.  A  SEMP  is  a  summary  management 
plan  covering  the  nature,  timing,  and  integration  of  all 
technical  development  activities.  The  SEMP  provides  program 
management  organization,  systems  engineering  direction,  control 
mechanisms,  and  personnel  to  be  used  throughout  development  to 
satisfy  cost,  performance,  and  schedule  objectives.  MIL-STD-499A 
was  not  included  in  the  FSD  contract  because  the  C-17  program  was 
considered  too  dynamic  to  make  comprehensive  management  plans. 
As  a  result,  Douglas  was  not  required  to  prepare  a  SEMP  or  its 
equivalent.  Instead,  Douglas  prepared  schedules  to  depict  the 
program's  planned  development.  We  recognize  that  programs  often 
change  as  they  are  developed,  but  we  believe  the  changes  must  be 
effectively  managed.  A  SEMP  would  have  provided  Douglas  a  more 
disciplined  approach  to  identifying  potential  program  risks, 
subsystem  interfaces,  and  possible  solutions  to  technical 
problems  in  the  system  development. 

Avionics  Integration  Test  Plan  and  Organization. 
Douglas  had  not  completed  the  planning  needed  to  ensure  that  all 
necessary  subsystems  will  be  tested  and  integrated  in  time  for 
the  aircraft’s  first  flight  in  June  1991.  Douglas  did  prepare  an 
avionics  integration  test  plan,  dated  December  1,  1989,  but  it 
was  incomplete.  The  plan  did  not  specify  all  required  tests  for 
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integration  of  two  or  more  subsystems.  The  plan  also  omitted 
system  level  tests  (tests  of  the  whole  avionics  system)  to  be 
made  in  the  flight  hardware  simulator  of  all  equipment 
functions.  According  to  SPO  officials,  the  test  plan  contained 
too  many  concurrent  tests  and  overlapping  tests,  and  allowed  too 
little  time  for  corrections  and  retesting.  Douglas  did  not 
establish  a  date  for  completing  the  avionics  integration  test 
plan.  In  1988,  the  SPO  withheld  $1.0  million  from  Douglas 
because  a  CDR  of  the  aircraft  determined  that  the  avionics  design 
was  not  complete  and  that  an  avionics  integration  test  plan  had 
not  been  prepared.  The  SPO  has  rejected  Douglas's  December  1989 
test  plan  as  a  basis  to  consider  the  avionics  design  and 
integration  plan  as  complete  and  to  pay  the  $1.0  million.  After 
completing  the  integration  plan,  Douglas  must  still  write  the 
procedures  for  the  individual  tests. 

SPO  and  Douglas  officials  stated  that  Douglas  had  not  developed 
the  test  plan  or  written  test  procedures  because  of  personnel 
shortages.  Attempts  to  hire  additional  people  with  the  proper 
skills  had  failed.  Douglas  planned  to  use  the  same  employees  to 
develop  and  perform  integration  tests  and  to  develop  system  level 
tests  of  all  functions.  Because  Douglas  lacked  experience  in 
system  integration,  SPO  officials  were  concerned  that  the  people 
writing  and  performing  integration  test  procedures  might  not  be 
fully  qualified  to  write  the  detailed  tests  needed  for  system 
integration.  Douglas  officials  told  us  that  they  were  getting 
more  experienced  staff  from  corporate  headquarters,  McDonnell 
Douglas  Corporation,  to  assist  in  the  effort. 

We  believe  that  integration  of  a  functional  avionics  system 
remains  a  major  challenge  to  the  successful  and  timely 
development  of  the  C-17  program.  Although  not  all  integration 
plans  and  test  procedures  demand  immediate  implementation, 
reasonable  estimates  of  the  amount  of  time  required  to  complete 
integration  cannot  be  made  until  detailed  planning  is  finished. 
In  our  opinion,  the  development  schedule  does  not  allow  much  time 
to  do  system  level  integration  if  the  first  flight  date  of 
June  1991  is  to  be  met.  Subcontractors  must  complete  development 
of  the  subsystems  and  deliver  them  to  Douglas  for  integration. 
Although  the  subsystems  may  work  properly  alone,  Douglas  must 
ensure  that  they  work  together  as  an  avionics  system.  An 
analysis  made  by  the  Aeronautical  Systems  Division,  AFSC, 
determined  that  48  percent  of  software  design  errors  are 
corrected  during  the  system  integration  phase.  Another  potential 
barrier  to  completing  system  level  testing  on  schedule  was  the 
completion  of  the  flight  hardware  simulator,  originally  scheduled 
for  September  1990. 

Contractor  Management  Organization.  Douglas  did  not 
establish  an  adequate  management  structure  at  the  outset  of  the 
C-17  program.  A  more  effective  organizational  structure  to 
better  manage  the  C-17  program  was  being  formed  late  in  the 
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program.  McDonnell  Douglas  Corporation  had  begun  transferring 
personnel  to  Douglas  from  other  corporate  components  to  provide 
the  experience  and  expertise  previously  not  available  at  Douglas. 
In  October  1989,  Douglas  established  a  general  manager's  office 
to  provide  full-time  oversight  of  the  EFCS  development.  We  were 
told  that  personnel  selected  to  staff  that  office  were  being 
transferred  to  Douglas  from  McDonnell  Douglas,  St.  Louis, 
Missouri,  where  they  had  helped  to  develop  the  EFCS  for  the  F-18 
aircraft.  In  another  change,  Douglas  recognized  that  the 
complexity  of  the  avionics  system  demanded  that  a  separate 
organization  be  made  responsible  for  avionics  development.  As  a 
result,  Douglas  established  the  position  of  general  manager  for 
the  C-17  avionics  system.  A  position  of  business  unit  manager 
for  avionics  system  integration  and  testing  was  also  established, 
but  as  of  February  2,  1990,  this  critical  position  had  not  been 
filled  officially.  As  of  the  same  date,  the  heads  of  2  of 
7  business  units  and  16  of  29  group  leaders  had  not  yet  been 
officially  appointed  to  manage  the  C-17  avionics  system 
development. 

In  February  1989,  in  a  move  affecting  all  of  the  programs  that 
Douglas  manages,  McDonnell  Douglas  directed  Douglas  to  make  a 
significant  change  to  its  management  structure.  The  change 
affected  both  military  and  commercial  programs  and  was  intended 
to  put  more  focus  on  individual  products  and  customers.  The 
reorganization  eliminated  four  levels  of  management  and  gave  the 
vice  president-general  manager  for  the  C-17  direct  control  of  the 
resources  working  on  the  program.  Although  the  reorganization 
may  have  long-term  beneficial  effects,  its  implementation 
disrupted  the  C-17  program,  because  supervisors  were  distracted 
during  the  reorganization,  and  not  all  supervisors  were  appointed 
immediately. 

Management  Approach  by  the  C-17  System  Program  Office. 
Because  Douglas  has  total  responsibility  for  the  contract,  the 
SPO  Program  Director  was  limited  in  directing  corrective  actions 
to  early  indicators  that  subcontractors  were  having  management 
problems.  The  C-17  Program  Director  informed  the  auditors  that 
he  did  not  seek  to  direct  the  actions  of  Douglas's  subcontractors 
so  that  the  SPO  would  not  relieve  Douglas  of  total  performance 
responsibility  and  shift  schedule  risks  from  Douglas  to  the 
Government.  To  prompt  Douglas  to  make  the  appropriate  decisions 
and  to  take  the  necessary  actions,  SPO  personnel  worked  closely 
with  Douglas  personnel  to  provide  oversight  of  program 
management.  For  example,  to  check  the  progress  of  the  program, 
the  Program  Director  required  Douglas  to  submit  reports  on 
"inchstones. "  In  addition,  SPO  personnel  actively  participated 
in  technical  interchange  meetings  with  Douglas  and  with  the 
subcontractors  to  ensure  requirements  were  being  communicated  and 
understood.  To  influence  Douglas  to  adhere  to  the  program 
schedule,  the  Program  Director  implemented  an  event-based 
contract  strategy.  In  November  1988,  the  SPO  modified  the  FSD 
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contract  to  make  procurements  of  C-17  aircraft  during  FY's  1989 
through  1992  contingent  on  Douglas  accomplishing  specified 
milestones.  Although  the  event-based  strategy  is  an  excellent 
method  to  ensure  that  procurements  are  made  only  when 
appropriate,  the  Program  Director  needed  to  redefine  the  event 
for  PY  1990  to  specify  the  minimum  assembly  work  to  be  completed 
on  the  T-l  (test  aircraft)  to  consider  the  T-l  to  be  assembled. 

Award  Fee  Evaluations.  We  believe  the  SPO's  formal 
evaluations  of  Douglas 1 s  management  highlighted  many  of  the 
problems  being  experienced  in  the  C-17  program  development.  The 
SPO  provided  formal  evaluations  to  Douglas  when  fixing  the  amount 
of  award  fees  earned  by  Douglas.  The  FSD  contract  included  a 
pool  of  $10  million  in  award  fees  to  provide  an  added  incentive 
to  effectively  develop  the  C-17.  The  planned  length  of  the  FSD 
contract  was  divided  into  seven  award  periods,  and  evaluation 
criteria  were  established  for  each  period  depending  on  the 
management  area  or  principle  the  Air  Force  wanted  to  emphasize. 
The  amount  of  the  award  fee  earned  for  each  period  depended  on 
how  well  Douglas  performed  in  each  evaluation  category.  Starting 
with  the  second  award  fee  period  (October  2,  1986,  to  July  1, 

1987),  the  SPO's  evaluation  has  included  comments  on  weaknesses 
in  Douglas's  subcontractor  management.  For  the  third  award  fee 
evaluation  period  (July  2,  1987,  to  April  1,  1988),  _  the  SPO 

criticized  Douglas's  subcontractor  management  more  specifically. 
Commenting  on  Douglas's  performance  during  the  fourth  award 
period  (April  2,  1988,  to  April  1,  1989),  the  SPO  stated: 

Tendency  to  award  subcontracts  and  assume 
subcontractor  will  work  the  problems  and 
deliver  on  time.  This  management  approach 
was  ill-advised  in  a  number  of  major 
subcontracts  ([with]  Delco,  Lockheed, 

LTV).  Definition  of  subcontractor 

requirements  and  formal  changes  (SCP*s 
[software  change  proposals])  are  not 
implemented  in  a  timely  manner. 

Sudden  and  sometimes  inadequately  explained 
schedule  slips  at  major  suppliers  indicate 
a  lack  of  depth  in  understanding  of 
progress  and  status  at  the  major/critical 
subcontractors.  The  capability  to 

anticipate  or  avoid  future  problems  has 
been  lacking* 

The  overall  rating  for  the  fourth  award  period  for  subcontractor 
management  was  30  percent  and  for  program  management,  25  percent, 
down  from  71  percent  and  85  percent,  respectively,  for  the  third 
award  period. 
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Progress  Payments.  Federal  Acquisition  Regulation, 
subpart  32.503-6,  authorizes  the  contracting  officer  to  withhold 
progress  payments  when  the  contractor  has  not  complied  with  all 
material  requirements  of  the  contract.  Before  withholding 
payments,  the  contracting  officer  must  notify  the  contractor  of 
the  intended  action  and  evaluate  the  effect  of  the  withheld 
payments  on  the  contractor's  financial  condition.  The  SPO 
withheld  one  progress  payment  of  $15  million  based  on  the  results 
of  a  CDR  of  the  C-17  aircraft  in  1988.  At  the  end  of  our  audit, 
the  SPO  had  released  $6.7  million  of  the  withheld  payment  based 
on  corrective  actions  taken  by  Douglas.  On  November  14,  1989, 
the  SPO's  Administrative  Contracting  Officer  (ACO)  issued  a 
letter  to  Douglas  stating  that  future  payments  may  be  withheld 
because  Douglas  had  not  met  program  schedule  inchstones  and 
milestones.  On  December  6,  1989,  Douglas  responded  that 
corrective  actions  had  been  taken  on  the  development  problems  and 
that  withheld  progress  payments  would  have  an  adverse  financial 
impact.  On  January  12,  1990,  the  ACO  informed  Douglas  that 
additional  withheld  payments  would  be  deferred  pending  further 
evaluation  of  Douglas's  progress  in  developing  the  aircraft  and 
of  Douglas's  financial  condition. 

Development  of  Hardware.  Although  we  concentrated  primarily 
on  problems  being  experienced  with  software  development,  during 
the  audit,  we  noted  some  difficulties  in  hardware  development. 
Host  significantly,  Douglas  did  not  accomplish  milestones  listed 
in  Special  Provision  H-ll  of  the  FSD  contract  to  join  the  wing  to 
the  fuselage  for  the  test  aircraft  (T-l)  by  July  31,  1989,  and  to 
begin  assembly  of  the  fourth  production  aircraft  by  January  31, 
1990.  Douglas  accomplished  the  wing-to-f uselage  join  in 
March  1990  and  rescheduled  the  assembly  start  of  the  fourth 
production  aircraft  to  July  1990.  An  August  18,  1989,  letter 
from  the  SPO  to  Douglas  stated  that  the  delay  in  joining  the  wing 
adversely  affected  the  whole  training  program  for  the  C-17.  The 
SPO  requested  Douglas  to  identify  all  schedule  dates  that  would 
be  affected  by  rescheduling  the  wing  join  and  to  submit  a  revised 
schedule  showing  when  Douglas  would  accomplish  all  contractual 
milestones.  In  addition,  the  SPO  asked  Douglas  to  propose 
compensation  to  the  Government  for  having  missed  the  milestone 
for  the  wing  join  and  to  provide  rationale  for  the  proposed 
compensation.  At  the  end  of  our  audit,  Douglas  had  not  responded 
to  the  SPO's  letter.  In  addition,  Douglas  had  not  provided  any 
compensation  to  the  Government  for  missing  the  contractual 
milestones. 

The  November  1989  Cost  Performance  Report  prepared  by  Douglas 
stated  that  problems  had  been  encountered  by  two  of  its 
subcontractors:  Lockheed  in  the  development  of  wing  components, 
and  LTV  on  the  development  of  horizontal  and  vertical  stabilizers 
and  refueling  panels.  After  the  completion  of  our  audit,  Douglas 
decided  to  replace  Lockheed  after  Lockheed  completes  the  wing 
components  through  the  sixth  C-17  production  aircraft.  Douglas 
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decided  to  obtain  other  subcontractors  for  the  wing  components 
because  Lockheed's  costs  were  increasing.  Lockheed  officials  had 
stated  that  additional  expenses  were  incurred  after  Douglas  began 
changing  requirements  for  the  wing  components. 

The  Hotline  complainant  had  expressed  concern  that,  at  a  very 
late  point  in  the  development  program,  Douglas  was  considering 
replacing  its  subcontractor.  Teledyne,  Inc.,  because  of  problems 
in  developing  hardware.  Douglas  officials  informed  us  that  the 
panel  being  designed  by  Teledyne  for  the  warning  and  caution 
system  was  incompatible  with  night-vision  goggles  and  exceeded 
touch-temperature  limitations.  Douglas  had  solicited  bids  from 
other  vendors  to  develop  a  suitable  panel  that  could  be 
successfully  incorporated  into  Teledyne's  warning  and  caution 
system.  Douglas  officials  stated  that  a  new  panel  could  not  be 
developed  and  installed  before  the  aircraft's  planned  first 
flight  in  June  1991.  However,  Douglas  did  not  expect  that  use  of 
Teledyne's  panel  would  adversely  affect  the  first  flight.  After 
discussions  with  the  SPO  and  Douglas,  the  auditors  concluded  that 
changing  subcontractors  for  the  panel  would  not  adversely  affect 
the  program  schedule.  Douglas  plans  to  install  the  panel  by  the 
sixth  production  aircraft  and  to  retrofit  the  initial  production 
aircraft  with  the  panel  at  no  additional  increase  to  the  target 
cost. 

Schedule  Realism.  Delays  in  the  development  of  software  and 
hardware  have  caused  the  date  of  the  first  flight  to  be 
postponed.  First  flight  was  originally  scheduled  to  occur  in 
August  1990,  but  in  August  1989,  the  SPO  Program  Director 
concluded  that  first  flight  would  not  occur  until  December 
1990.  Following  a  DAB  review  in  October  1989,  the  first  flight 
was  changed  officially  to  June  1991.  In  our  opinion,  the 
June  1991  date  will  provide  additional  needed  time,  but  the 
schedule  is  still  success-oriented  with  little  time  available  for 
unforeseen  delays.  Douglas  has  identified  the  software  required 
for  first  flight  and  is  placing  emphasis  on  developing  that 
software  to  enable  the  first  flight  to  be  made  on  time.  However, 
SPO  and  Douglas  officials  told  us  that  problems  being  experienced 
with  hardware  development  made  it  impossible  to  predict  whether 
the  hardware  or  the  software  would  be  completed  first.  SPO 
officials  told  us  that  the  EFCS  and  the  mission  computer  were 
still  considered  to  be  high  risk  developments.  Early  lapses  by 
Douglas  in  systems  engineering  adversely  affected  subcontractors 
Honeywell  and  Delco  and  may  yet  affect  subsystems  being  developed 
by  other  subcontractors.  Because  Douglas  lacks  personnel 
experienced  in  integrating  complex  avionics  systems  and  because 
Douglas  has  not  prepared  integration  test  planning  in  sufficient 
detail,  the  needed  integration  may  not  be  accomplished  in  time 
for  the  June  1991  first  flight.  Douglas  has  made  and  is 
continuing  to  make  changes  to  the  management  of  the  program 
intending  to  correct  development  difficulties.  These  changes 
improve  the  chances  that  the  program  will  be  developed  on  time. 
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but  because  the  changes  are  being  made  late  in  the  program  and 
because  they  have  not  yet  proved  to  be  successful,  further  delays 
in  the  program  schedule  may  yet  occur. 

RECOMMENDATIONS  FOR  CORRECTIVE  ACTION 

1.  We  recommend  that  the  Under  Secretary  of  Defense  for 
Acquisition  establish  procurement  policy  applicable  to  contracts 
for  major  acquisition  programs  that  assign  total  performance 
responsibility  to  contractors.  At  a  minimum,  this  policy  should 
require  that  a  systems  engineering  management  plan  (SEMP),  or  its 
equivalent,  be  required  in  full-scale  development  contracts  and 
that  the  SEMP  be  updated  when  determined  necessary  by  the  Program 
Director . 

2.  We  recommend  that  the  Program  Director  for  the  C-17  aircraft 
System  Program  Office,  Air  Force  Systems  Command: 

a.  Set  a  completion  date  for  Douglas  Aircraft  Company 
to  finalize  and  issue  an  avionics  integration  test  plan 
addressing  integration  testing  of  multiple  subsystems  and  system- 
level  testing  of  all  equipment  functions  to  be  performed  before 
the  first  flight  of  the  C-17  aircraft. 

b.  Require  Douglas  Aircraft  Company  to  prepare  a 
revised  schedule  showing  new  program  milestone  dates. 

c.  Obtain  compensation  to  the  Government  from  Douglas 
Aircraft  Company  for  the  missed  contract  milestones  relating  to 
the  wing-to-fuselage  join  and  the  assembly  start  of  the  fourth 
production  aircraft. 


MANAGEMENT  COMMENTS 


The  Director  of  Defense  Research  and  Engineering  nonconcurred 
with  Recommendation  B.I.,  because  broad  guidance  on  systems 
engineering  is  already  available  in  the  draft  DoD  Instruction 
5000.2,  "Defense  Acquisition  Management  Policies  and  Procedures," 
and  because  it  was  not  clear  that  avionics  development  and 
integration  difficulties  would  have  been  avoided  even  if 
additional  guidance  had  been  provided.  He  also  stated  that 
requiring  a  SEMP  in  every  full-scale  development  contract  may  not 
be  prudent  policy.  Management's  complete  comments  are  provided 
in  Appendix  C. 

The  Air  Force  nonconcurred  in  Recommendation  B.2.a.  through 
B.2.c.  Regarding  Recommendation  B.2.a.,  the  Air  Force  stated 
that  C-17  avionics  planning  is  an  evolutionary  process.  Douglas 
submitted  a  revised  C-17  Avionics  Integration  Test  Plan  in 
April  1990  and  again  in  July  1990.  Although  the  Air  Force 
nonconcurred  with  Recommendation  B.2.b.,  the  Air  Force  stated 
that  Douglas  has  "rebaselined  their  schedules  to  the  new  program 
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milestones."  Concerning  Recommendation  B.2.C.,  the  Air  Force 
stated  that  consideration  for  the  overall  schedule  slip  was 
expected  to  be  obtained  from  Douglas  by  August  30,  1990,  and  the 
process  to  obtain  compensation  was  begun  before  the  audit  was 
initiated.  A  complete  text  of  management  comments  is  provided  in 
Appendix  D. 


AUDIT  RESPONSE  TO  MANAGEMENT  COMMENTS 

Regarding  Recommendation  B.I.,  we  agree  with  management  that 
draft  DoD  Instruction  5.000.2  provides  guidance  for  systems 
engineering  for  acquisition  programs.  Because  draft 
Instruction  5000.2  requires  the  program  office  to  obtain  a  SEMP 
from  the  contractor,  we  believe  that  management  agrees  that 
policy  should  be  established  to  require  contractors  to  submit  a 
SEMP.  However,  the  proposed  policy  in  the  Instruction  does  not 
require  that  the  SEMP  be  part  of  the  FSD  contract  or  that  it  be 
updated  when  required.  Effective  development  of  complex  weapon 
systems  demands  that  sound  planning  be  initiated  at  the  beginning 
of  the  program  and  be  updated  as  the  program  develops.  Although 
we  acknowledge  that  the  avionics  and  integration  difficulties 
encountered  in  the  C-17  program  might  not  have  been  avoided  with 
a  SEMP,  we  believe  a  well-conceived  plan  would  have  caused  the 
problems  to  be  identified  earlier  in  the  program.  Accordingly, 
we  request  that  management  reevaluate  its  position  on  the 
recommendation  in  response  to  the  final  report. 

Recommendation  B.2.a.  would  require  Douglas  to  determine  the 
feasibility  of  accomplishing  all  required  C-17  avionics 
integration  testing  within  the  program  schedule.  Until  the  test 
plans  and  procedures  are  completed,  the  Air  Force  cannot  be 
assured  that  all  necessary  testing  can  be  accomplished  to  meet 
program  milestones.  Establishing  a  deadline  to  complete  the 
avionics  integration  test  plan  would  also  force  Douglas  to 
finalize  the  definition  of  all  requirements  and  to  complete  the 
detail  design  of  all  subsystems  needed  for  first  flight. 
Although  Douglas  submitted  an  incomplete  plan  and  two  revisions, 
the  Air  Force  still  had  not  paid  the  $1.0  million  withheld  in 
1988.  Consequently,  we  ask  that  management  reevaluate  its 
position  on  the  recommendation  in  response  to  the  final  report. 

Although  the  Air  Force  nonconcurred  in  Recommendation  B.2.b.,  the 
action  taken  by  the  Air  Force  to  obtain  a  revised  program 
schedule  meets  the  intent  of  the  recommendation.  Accordingly, 
additional  management  comments  on  the  recommendation  are  not 
required. 

Recommendation  B.2.c.  would  require  Douglas  to  provide 
compensation  for  the  missed  contractual  milestones.  The 
recommendation  was  made  to  add  impetus  to  the  Air  Force's  efforts 
to  obtain  compensation  from  Douglas.  During  the  audit,  we  were 
informed  that  Douglas  had  not  responded  to  an  August  18,  1989, 
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letter  from  the  SPO  requesting  compensation  for  missed 
milestones.  We  were  informed  on  October  18,  1990,  that  a 
contract  modification  providing  compensation  to  the  Government 
was  signed  on  September  24,  1990.  Therefore,  we  ask  that  the  Air 
Force  provide  in  its  comments  on  the  final  report  the  type  and 
amount,  if  monetary,  of  the  compensation  obtained. 
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SECTION  H— 111  OF  THE  C-17  FULL-SCALE  DEVELOPMENT  CONTRACT 


SECTION  H  (Continued) 

111.  PREREQUISITE  TO  OUTYEAR  PRODUCTION  AWARD 

1.  Annual  long  lead  for  each  production  buy  will  be  awarded  on  or  before 
31  Oan  of  each  year. 

2.  Douglas  Aircraft  Company  will  commit  to  an  aircraft  delivery  schedule  as 
part  of  each  long  lead  award. 

3.  Notwithstanding  any  other  provision  of  this  contractor  or  any  successor 
contract  hereto,  the  contractor  agrees  that  the  expiration  date  of  the  FY89 
production  option  under  this  contract  (CLINs  0013  thru  0018);  the  effective 
date  of  the  successor  production  contract  for  FY90  (Lot  III);  and,  the 
expiration  dates  of  the  production  options  for  FY91  (Lot  IV)  and  FY92  (Lot  V) 
under  the  said  successor  contract,  are  conditioned  on  the  completion  of  the 
following  respective  milestones  by  the  following  respective  dates: 


APPROXIMATE 

COMPLETION 

FY  BUY  MILESTONES  DATE _ 

89  PCO  Determination  of  Mission  Computer  Mar  89 

CDR  Complete 

90  This  milestone  will  be  accomplished  when  Oan  90 

T-l  moves  out  of  assembly  position  1  and 

the  PCO  determines  that  any  remaining  assembly 
work,  can  be  completed  without  significant 
disruption  to  planned  ground  and  flight  test 
efforts. 

91  PCO  determination  of  the  First  Flight  of  Oct  90 

of  T-l  and  P-1,  whichever  shall  last  occur 

92  P-5  Completion  of  FCA/PCA  &  Delivery  Sep  91 

(signed  OD  250) 

Should  the  contractor  not  complete  the  above  milestones  In  accordance  with 
this  schedule,  the  contractor  hereby  agrees  to  grant  to  the  Air  Force  an 
extension  of  the  expiration  date  of  the  FY89  production  option  under  this 
contract  (CLINs  0013  thru  0018)  until  the  date  30  days  after  the  Mission 
Computer  COR  is  acknowledged  by  the  Air  Force  to  be  completed;  an  extension  of 
one  day  for  each  day  completion  of  any  above  milestone  Is  delayed  In  the 
effective  date  of  the  successor  production  contract  for  FY90  production 
(Lot  III);  and,  an  extension  of  one  day  for  each  day  completion  of  any  above 
milestone  Is  delayed  Is  the  expiration  dates  of  the  production  options  for 
FY91  (Lot  IV)  and  FY92  (Lot  V)  under  the  successor  production  contract. 
Further,  the  contractor  agrees  that  any  such  extension  will  not  also  extend 
the  contract  delivery  schedule  of  the  CLINs  of  this  contract  or  any  successor 
contract  which  are  affected  thereby  and  will  not  require  the  obligation  by  the 
Air  Force  of  any  additional  long  lead  funds  on  this  contract  or  any  successor 
contract. 
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SECTION  H— 111  OF  THE  C-17  FULL-SCALE  DEVELOPMENT  CONTRACT  (CONT'D) 


SECTION  H  (Continued) 

4.  Nothing  In  this  clause  precludes  the  Government  from  exercising  any  of 
the  FY  buys,  as  shown  In  paragraph  3  above,  before  the  milestone  Is  complete, 
if  It  so  elects  to.  However,  no  option  will  be  exercised  under  this  contract, 
nor  will  award  of  any  successor  contract  be  made,  nor  will  any  production 
option  under  such  a  successor  contract  be  exercised,  until  the  availability  of 
funds  for  obligation  In  the  Fiscal  Year  In  question  has  been  certified. 

5.  Upon  evidence  of  completion  of  a  milestone,  the  Government  has  30  days 
to  exercise  the  corresponding  FY  production  buy.  If  It  so  elects  to. 

6.  The  milestone  for  FY90  must  be  met  before  the  next  production  contract 
(Lot  III)  Is  awarded. 

7.  Upon  award  of  the  next  production  contract  (Lot  III),  this  clause  will 
be  transferred  to  the  new  contract  to  extend  coverage  to  the  FY91  and  FY92 
production  option. 
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STATUS  OF  CORRECTIVE  ACTION  PLANS  FOR 
MI SSION  COMPUTER  SOFTWARE  CRITICAL  DESIGN  REVIEW  ITEMS 
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STATPS  OF  CORRECTIVE  ACTION  PLANS  FOR 
MISSION  COMPUTER  SOFTWARE  CRITICAL  DESIGN  REVIEW  ITEMS  (Continued) 
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DIRECTOR.  OF  DEFENSE  RESEARCH  AND  ENGINEERING 
WASHINGTON,  DC  20301-3010 

August  20,  1990 


MEMORANDUM  FOR  THE  INSPECTOR  GENERAL 

SUBJECT:  Draft  Audit  Report  on  Selected  Acquisition  Actions  on  the 
C-17  Aircraft 


This  responds  to  your  June  14,  1990,  memorandum  requesting 
comments  on  the  subject  draft  report. 

Section  A  -  Recommendation  1. 

I  do  not  concur  with  the  recommendation  that  the  Conventional 
Systems  Committee  (CSC)  of  the  Defense  Acquisition  Board  (DAB)  be 
staffed  with  "at  least  two  members  having  expertise  in  computer 
software  developments . "  The  report  implies  that  the  Office  of  the 
Deputy  Director  of  Defense  Research  and  Engineering  (Research  and 
Advanced  Technology) ,  with  only  one  seat  on  the  CSC,  is  "unable  to 
send  individuals  with  expertise  in  software,  plus  all  other  system 
development  areas,  to  the  Committee  meetings."  This  is  simply  not 
the  case.  The  CSC  has  always  maintained  a  liberal  policy  regarding 
attendance  at  meetings  by  supporting  staff  members.  No  principal 
staff  member  is  denied  the  opportunity  to  bring  staff  members  with 
relevant  expertise  to  CSC  meetings,  particularly  when  that  expertise 
is  needed  for  a  full  understanding  of  a  technical  issue. 

I  recognize  that  the  intent  of  this  recommendation  is  to  help 
strengthen  oversight  of  software  issues  in  major  system  reviews.  I 
support  that  objective,  and  agree  with  your  appraisal  that  "steps 
already  initiated  by  the  Department  to  improve  the  oversight  of 
software  development ....  will  help  ensure  that  software  developments 
are  properly  addressed  by  the  DAB  and  its  subordinate  committees." 

In  light  of  the  fact  that  CSC  administrative  procedures  allow 
for  sufficient  representation  of  subject  matter  experts,  and  actions 
to  strengthen  software  oversight  were  identified  during  the  course  of 
the  audit,  this  recommendation  should  be  withdrawn. 

SECTION  B  -  Recommendation  1. 

I  do  not  concur  with  the  recommendation  that  a  system  engineer¬ 
ing  management  plan  (SEMP) ,  or  equivalent  document,  be  required  in 
all  full-scale  development  contracts  for  major  acquisition  programs. 
The  draft  DODD  5000.1  provides  broad  guidance  on  systems  engineering 
and  technical  planning  considerations.  Within  the  Air  Force,  systems 
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Final  Report 
Page  No. 


engineering  management  is  addressed  in  MIL-STD-499,  which  is  cur¬ 
rently  being  revised  for  adoption  by  the  other  Services  as  a  Tri-Ser¬ 
vice  MIL-STD.  Thus,  the  acquisition  policy  guidance  and  implementa¬ 
tion  standards  exist  for  use  when  it  is  determined  to  be  in  the 
government's  best  interest  to  employ  that  level  of  oversight  in 
program  management. 

It  is  not  clear  that  the  schedule  inpact  of  avionics  development 
and  integration  difficulties  encountered  in  the  early  stages  of  C-17 
development  would  have  been  prevented  or  significantly  reduced  had  a 
SEMP  been  required.  In  that  same  light,  there  is  insufficient 
evidence  to  support  the  conclusion  that  requiring  a  SEMP  in  every 
full-scale  development  contract  is  prudent  policy.  This  should 
remain  an  option  subject  to  decision  on  a  case-by-case  basis. 
Accordingly,  this  recommendation  should  be  withdrawn. 

Internal  Controls 


The  discussion  of  "non-material  weaknesses"  in  the  context  of 
internal  controls  on  pages  iii,  5  and  55  of  the  draft  report  has  been 
the  source  of  considerable  confusion.  I  recognize  there  is  a 
requirement  to  state  whether  or  not  "material  internal  control 
weaknesses"  were  identified  during  the  course  of  the  audit.  Since  no 
material  internal  control  weaknesses  were  identified,  I  recommend 
that  the  discussions  of  "non-material  weaknesses"  be  deleted  or 
suitably  modified,  as  per  recent  discussions  between  our  staffs. 

This  will  prevent  the  merits  of  the  findings  being  obscured  by  the 
question  of  their  relevance  to  statutorily  defined  internal  control 
guidelines . 
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DEPARTMENT  OF  THE  AIR  FORCE 

WASHINGTON  DC  20330-1000 


OF  THE  ASSISTANT  SECRETARY 


AUG  1  7  1990 


MEMORANDUM  FOR  ASSISTANT  INSPECTOR  GENERAL  FOR  AUDITING 
OFFICE  OF  THE  INSPECTOR  GENERAL 
DEPARTMENT  OF  DEFENSE 

SUBJECT:  DoD( IG)  Draft  Audit  Report  on  Selected  Acquisition 

Actions  on  the  C-17  Aircraft,  June  14,  1990,  DoD(IG) 
Project  No.  0RA-8001  -  INFORMATION  MEMORANDUM 

This  memorandum  Is  In  reply  to  the  request  for  findings  and 
recommendations  made  In  the  subject  draft  DoD( IG)  report. 
Responses  are  included  for  Recommendations  A. 2. a.  and  b.  and 
B.2 .a. ,b. .and  c..  Responses  are  not  included  for 
Recommendations  A.1.  and  B.l.  since  these  recommendations  are 
directed  to  the  Under  Secretary  of  Defense  for  Acquisition. 


The  Air  Force  will  continue  to  employ  the  appropriate 
management  emphasis  to  each  area  addressed  In  the  final  report 
as  well  as  our  commitment  to  the  timely  completion  of  the 


agreed-upon  actions. 


1  Attachment 
Air  Force  Comments 
Draft  DoD(IG)  Report 
Project  No.  0RA-8001) 


DANIELS.  RAK 
Deputy  Assistant  Secretaiy 
(Acquisition) 
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AIR  FORCE  COMMENTS 
DoD(IG)  Draft  Report 
(Project  No.  0RA-8001 ) 

FINDING  A.  Mission  Computer  Software  Critical  Design  Review: 

The  C-1 7  System  Program  Office  (SPO)  exercised  the  FY  1989 
contract  option  to  buy  four  C-17  aircraft  before  Douglas 
Aircraft  Company  (Douglas)  completed  the  required  critical 
design  review  (CDR)  of  all  mission  computer  software. 

Completion  of  the  CDR,  a  prerequisite  to  exercising  the  option, 
was  required  by  November  1988  modification  of  the  full-scale 
development  (FSD)  contract.  The  C-17  Program  Director 
considered  the  CDR,  made  In  April  1989,  to  be  adequate  and 
complete,  even  though  Douglas  had  completed  the  detail  design 
for  only  about  60  percent  of  all  software  In  the  mission 
computer.  The  C-17  program  development  has  already  experienced 
major  delays,  and  unless  all  required  mission  computer  software 
is  developed  before  the  C-17  passes  from  the  developmental  test 
and  evaluation  phase  to  the  operational  test  and  evaluation 
phase,  the  C-17  program  schedule  will  be  further  delayed. 


Response 

Non-concur .  This  finding  implies  the  FY  1989  contract 
option  for  four  C-17  production  aircraft  was  prematurely 
awarded.  We  disagree.  The  Intent  of  the  event  of  the  Event 
Based  Contract  Award  strategy,  as  incorporated  into  the  C-17 
FSED  contract  In  November  1988,  was  to  further  Incentivlze  DAC's 
schedule  performance.  The  mission  computer  CDR  was  selected  as 
the  FY  1989  option  milestone  to  emphasize  Air  Force  concern  over 
avionics  development  progress.  The  CDR  was  completed  per  the 
SPO-DAC  November  1988  CDR  Definition  agreement,  which  Included 
all  requirements  and  top  level  design  for  the  system.  In  fact, 
award  of  the  contract  was  delayed  for  more  than  three  months 
after  the  CDR  itself  to  ensure  closure  of  action  Items  and  to 
gain  a  written  schedule  commitment  from  both  DAC  and  Delco  on 
mission  computer  development.  That  schedule  is  being  met.  The 
IG  team  stated  during  their  debriefing  that  their  point  of 
reference  on  the  CDR  was  that  of  an  "Academic  CDR"  not  one  that 
met  the  C-17  Program  (SPO-DAC)  agreement.  Mission  computer 
development  is  on  track  to  meet  program  schedule  and  user  needs. 
DAC  and  the  SPO  have  always  planned  to  complete  mission  computer 
development  before  the  completion  of  development  test  and 
evaluation. 


RECOMMENDATION  A. 2. a; 


Require  completion  of  the  mission  computer  software 
critical  design  review,  planned  for  October  1990,  as  a 
precondition  to  awarding  the  Lot  III  production  contract  to 
procure  four  aircraft  for  FY  1990. 

APPENDIX  D 
Page  2  of  5 


40 


Response 


Non-concur.  DAC  and  the  SPO  consider  the  mission  computer 
CDR  to  be  completed,  per  the  bilateral  Event  Based  Contract 
Award  definition.  DAC  met  all  requirements.  Since  our 
agreement  is  bilateral,  the  government  has  no  cause  and  the 
contractor  has  no  reason  to  reopen  the  Event  Based  Contract. 

RECOMMENDATION  A.2.b. 

Specify  the  minimum  assembly  required  to  consider  the  test 
aircraft  (T-1)  assembled  for  the  milestone  event  for  Lot  III. 

Response 

Non-concur.  The  SPO  and  DAC  have  reached  agreement  on  a 
more  precise  definition  of  the  T-1  assembly  complete  milestone. 
This  definition  has  been  formalized  In  a  memorandum  of 
understanding,  signed  by  DAC,  and  transmitted  to  the  SPO  by 
letter  on  14  June  1990.  This  Is  a  normal  part  of  the  Event 
Based  Contract  Award  process  and  was  In  no  way  driven  or 
Influenced  by  the  DoD  IG  report.  No  further  action  is  required. 

FINDING  B:  Engineering  Planning  and  Subcontractor  Management: 

Douglas  had  not  fully  accomplished  technical  requirements 
to  meet  contractual  milestones  for  the  development  of  the  C-17 
aircraft  and  had  not  prepared  an  adequate  Integration  test  plan 
for  the  development  and  Integration  of  software  for  the  avionics 
system.  The  total  performance  responsibility  contract  that  the 
Air  Force  has  with  Douglas  does  not  specify  all  management  plans 
necessary  for  the  successful  development  of  the  C-17  program  and 
restricts  the  action  of  the  SPO  to  ensure  that  milestones  are 
accomplished.  Delays  In  the  development  of  the  aircraft  have 
caused  the  Air  Force  to  postpone  major  program  milestones,  and 
further  delays  may  occur. 

Response 

Non-concur.  See  below. 

RECOMMENDATION  B.2.a. 


Set  a  completion  date  for  Douglas  Aircraft  Company  to 
finalize  and  Issue  an  avionics  Integration  test  plan  addressing 
integration  testing  of  multiple  subsystems  and  system- 1  eve  I 
testing  of  all  equipment  functions  to  be  performed  before  the 
first  flight  of  the  C-17  aircraft. 
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Response 

Non-concur.  The  C-17  avionics  planning  Is  an  evolutionary 
process.  C-17  avionics  development  plans  have  been  In  existence 
since  the  first  submittal  by  the  contractor  In  1984.  The  most 
recent  version  of  the  C-17  Avionics  Integration  Test  Plan  was 
submitted  by  DAC  In  April  1990.  The  SPO  has  worked  closely  with 
DAC  to  assure  the  document  remains  an  effective  management  tool. 

An  updated  version  was  received  In  July  1990.  This  process  Is 
normal  and  not  required  as  a  result  of  the  DOD  IG  Inspection. 

RECOMMENDATION  B.2.b 

Require  Douglas  Aircraft  Company  to  prepare  a  revised  schedule 
showing  new  program  milestone  dates. 

Response 

Non-concur.  DAC  has  rebase  lined  their  schedules  to  the  new 
program  ml lestones  as  part  of  the  formal  program  rebasel Ine 
Initiated  by  the  SPO  In  the  late  summer  of  1989.  The  next  formal 
submittal  of  the  Summary  Program  Master  Schedule  was  submitted  in 
July  1990  and  reflects  the  rebasel l ned  schedule. 

RECOMMENDATION  B.2.C. 

Obtain  compensation  to  the  Government  from  Douglas  Aircraft 
Company  for  the  missed  contract  milestones  relating  to  the  wlng- 
to-fuselage  Join  and  the  assembly  start  of  the  fourth  production 
aircraft . 

Response; 

Non-Concur.  The  missed  contract  milestones  were  part  of  an 
overall  C-17  schedule  slip  affecting  contract  dates  for  FSED,  the 
1988  production  option,  and  the  1989  production  option.  A  revised 
schedule  and  consideration  were  negotiated  In  June  1990.  The 
contract  modification  for  this  agreement  Is  currently  In  process 
and  expected  to  be  Issued  by  30  August  1990.  Consideration  was 
negotiated  for  the  overall  slip  and  is  not  segregable  by 
Individual  milestone.  This  process  was  begun  before  the  DOD  IG 
Initiated  Its  audit. 

GENERAL  COMMENTS: 

In  addition  to  the  above  comments,  we  recommend  the  following 
clarifications  to  subject  draft  report. 

a.  Page  7,  Independent  Verification  &  Validation  (IV&V). 
Since  the  DOD  IG  Is  using  the  report  to  update  the  status  of  the 
previous  reports,  some  clarification  Is  necessary.  The  last 
paragraph  In  the  section  dealing  with  report  89-059  leads  to  the 
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Impression  that  Douglas  Initiated  Internal  IV&V  In  deference  to 
the  Air  Force's  nonconcurrence  that  an  IV&V  would  be  beneficial. 
This  is  Inconsistent  with  the  facts  and  that  which  the  DOD  IG 
stated  on  page  23  of  the  89-059  report.  In  the  report,  they 
correctly  state  that  the  Air  Force  initiated  the  activities  that 
resulted  in  Douglas  implementing  an  Internal  IV&V  program  at  no 
cost  to  the  Air  Force  as  the  referenced  report  states. 

b.  Page  15,  1 ine  2.  Insert  "four"  between  "in"  and 
“Increments"  to  clarify  which  increments  are  being  referred  to. 

c.  Page  16,  lines  10-14.  This  sentence  should  be  rewritten 
for  clarity.  DAC  did  agree  to  a  post-CDR  software  review,  but  not 
until  after  the  November  1988  contract  modification. 

d.  Page  17: 

(1)  Lines  13-16.  This  sentence  Implies  requirements  and 
design  were  not  sufficiently  complete  for  the  CDR .  Actually,  they 
were  not  sufficient  for  completion  of  coding.  During  the  CDR, 
insufficiencies  were  identified  and  corrective  action  plans 
developed  and  approved  by  DAC  with  the  SPO's  concurrence. 

(2)  Lines  16-21.  The  development  of  mission  computer 
software  was  identified  as  a  high-risk  item  long  before  the  CDR. 
This  sentence  implies  otherwise. 
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SUMMARY  OF  POTENTIAL  MONETARY  AND  OTHER 
BENEFITS  RESULTING  FROM  AUDIT 


Recommendation 

Reference 


A.l. 


A. 2 .a. 


A.2.b. 


B.l. 


B.2 .a. 


B.2.b. 


B  •  2  •  c  • 


_ Description  of  Benefit _  Type  of  Benefit 

Economy  and  Efficiency.  Improve  the  Nonmonetary 
OSD  oversight  process  by  requiring 
individuals  with  software  expertise 
to  attend  meetings  of  the  Conven¬ 
tional  Systems  Committee  of  the 
Defense  Acquisition  Board. 

Economy  and  Efficiency.  Ensure  Nonmonetary 

completion  of  the  mission  computer 

software  design  review  by  making  the 

review  a  precondition  to  buying 

C-17  aircraft  in  FY  1990. 


Economy  and  Efficiency.  Ensure  the  Nonmonetary 
FY  1990  procurement  is  appropriately 
made  by  clarifying  the  preconditions 
to  buying  C-17  aircraft. 

Economy  and  Efficiency.  Protect  the  Nonmonetary 

Government's  interest  by 

establishing  policy  for  total 

performance  responsibility 

contracts. 


Economy  and  Efficiency.  Establish  Nonmonetary 

integration  test  requirements  and 
time  frames  by  requiring  a  final 
integration  test  plan. 

Economy  and  Efficiency.  Determine  Nonmonetary 

the  program  development  schedule  by 
requiring  a  revised  milestone  schedule. 


Economy  and  Efficiency.  Obtain  Nonmonetary  * 

compensation  for  missed  contrac¬ 
tual  milestones. 


*  We  are  not  claiming  monetary  benefits  for  any  compensation  because  the  Air 
Force  began  efforts  to  obtain  compensation  before  the  audit  was  initiated. 
The  recommendation  was  made  to  add  impetus  to  the  Air  Force's  efforts  to 
obtain  compensation,  because  Douglas  had  not  responded  to  a  request  by  the  Air 
Force  for  compensation  for  missed  contractual  milestones. 


45 


APPENDIX  E 


This  page  was  left  out  of  original  document 


This  page  was  left  out  of  original  document 


This  page  was  left  out  of  original  document 


AUDIT  TEAM  MEMBERS 


William  P.  Thomas,  Director,  Readiness  and  Operational  Support 

Ronald  R.  Porter,  Program  Director 

John  Mundell,  Project  Manager 

William  Hopple,  Team  Leader 

Evelyn  Klemstine,  Team  Leader 

John  A.  Galloway,  Auditor 

Wei  K.  Chang,  General  Engineer 

Gregory  Donnellon,  Industrial  Specialist 

David  L.  Leising,  Contract  Specialist 
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FINAL  REPORT  DISTRIBUTION 


Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  for  Acquisition 
Director  of  Defense  Research  and  Engineering 
Assistant  Secretary  of  Defense  (Production  and  Logistics) 
Assistant  Secretary  of  Defense  (Program  Analysis  and  Evaluation) 
Assistant  Secretary  of  Defense  (Public  Affairs) 

Department  of  the  Air  Force 

Secretary  of  the  Air  Force 

Assistant  Secretary  of  the  Air  Force  (Financial  Management  and 
Comptroller) 

Air  Force  Audit  Agency 

Other  Defense  Activities 

Defense  Logistics  Agency 

Commandant,  Industrial  College  of  the  Armed  Forces 
Defense  Logistics  Studies  Information  Exchange 

Non-Defense  Activities 


Office  of  Management  and  Budget 
U.S.  General  Accounting  Office, 

NS I AD  Technical  Information  Center 

Congressional  Committees: 

Senate  Subcommittee  on  Defense,  Committee  on  Appropriations 
Senate  Committee  on  Armed  Services 
Senate  Committee  on  Governmental  Affairs 

Senate  Ranking  Minority  Member,  Committee  on  Armed  Services 
House  Committee  on  Appropriations 

House  Subcommittee  on  Defense,  Committee  on  Appropriations 
House  Ranking  Minority  Member,  Committee  on  Appropriations 
House  Committee  on  Armed  Services 
House  Committee  on  Government  Operations 
House  Subcommittee  on  Legislation  and  National  Security, 
Committee  on  Government  Operations 
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INTERNET  DOCUMENT  INFORMATION  FORM 


A  .  Report  Title:  Selected  Acquisition  Actions  on  the  C-17  Aircraft 

B.  DATE  Report  Downloaded  From  the  Internet:  08/14/00 


C.  Report’s  Point  of  Contact:  (Name,  Organization,  Address,  Office 
Symbol,  &  Ph  #):  OAIG-AUD  (ATTN:  AFTS  Audit  Suggestions) 

Inspector  General,  Department  of  Defense 
400  Army  Navy  Drive  (Room  801) 

Arlington,  VA  22202-2884 


D.  Currently  Applicable  Classification  Level:  Unclassified 

E.  Distribution  Statement  A:  Approved  for  Public  Release 

F„  The  foregoing  information  was  compiled  and  provided  by: 
DTIC-OCA,  initials: _ VM _ Preparation  Date  08/14/00 


The  foregoing  information  should  exactly  correspond  to  the  Title,  Report  Number,  and  the  Date 
the  accompanying  report  document.  If  there  are  mismatches,  or  other  questions,  contact  the 
above  OCA  Representative  for  resolution. 


